Incorporating an Early Detection System Into Routine Clinical Practice in Two Community Hospitals

Video-Assisted vs Robotic-Assisted Lung Lobectomies for Operating Room Resource Utilization and Patient Outcomes

Importance

With increased use of robots, there is an inadequate understanding of minimally invasive modalities' time costs. This study evaluates the operative durations of robotic-assisted vs video-assisted lung lobectomies.

Objective

To compare resource utilization, specifically operative time, between video-assisted and robotic-assisted thoracoscopic lung lobectomies.

Design, Setting, and Participants

This retrospective cohort study evaluated patients aged 18 to 90 years who underwent minimally invasive (robotic-assisted or video-assisted) lung lobectomy from January 1, 2020, to December 31, 2022, with 90 days' follow-up after surgery. The study included multicenter electronic health record data from 21 hospitals within an integrated health care system in Northern California. Thoracic surgery was regionalized to 4 centers with 14 board-certified general thoracic surgeons.

Exposures

Robotic-assisted or video-assisted lung lobectomy.

Main Outcomes and Measures

The primary outcome was operative duration (cut to close) in minutes. Secondary outcomes were length of stay, 30-day readmission, and 90-day mortality. Comparisons between video-assisted and robotic-assisted lobectomies were generated using the Wilcoxon rank sum test for continuous variables and the χ2 test for categorical variables. The average treatment effects were estimated with augmented inverse probability treatment weighting (AIPTW). Patient and surgeon covariates were adjusted for and included patient demographics, comorbidities, and case complexity (age, sex, race and ethnicity, neighborhood deprivation index, body mass index, Charlson Comorbidity Index score, nonelective hospitalizations, emergency department visits, a validated laboratory derangement score, a validated institutional comorbidity score, a surgeon-designated complexity indicator, and a procedural code count), and a primary surgeon-specific indicator.

Results

The study included 1088 patients (median age, 70.1 years [IQR, 63.3-75.8 years]; 704 [64.7%] female), of whom 446 (41.0%) underwent robotic-assisted and 642 (59.0%) underwent video-assisted lobectomy. The median unadjusted operative duration was 172.0 minutes (IQR, 128.0-226.0 minutes). After AIPTW, there was less than a 10% difference in all covariates between groups, and operative duration was a median 20.6 minutes (95% CI, 12.9-28.2 minutes; P < .001) longer for robotic-assisted compared with video-assisted lobectomies. There was no difference in adjusted secondary patient outcomes, specifically for length of stay (0.3 days; 95% CI, -0.3 to 0.8 days; P = .11) or risk of 30-day readmission (adjusted odds ratio, 1.29; 95% CI, 0.84-1.98; P = .13). The unadjusted 90-day mortality rate (1.3% [n = 14]) was too low for the AIPTW modeling process.

Conclusions and Relevance

In this cohort study, there was no difference in patient outcomes between modalities, but operative duration was longer in robotic-assisted compared with video-assisted lung lobectomy. Given that this elevated operative duration is additive when applied systematically, increased consideration of appropriate patient selection for robotic-assisted lung lobectomy is needed to improve resource utilization.

The algorithm journey map: a tangible approach to implementing AI solutions in healthcare

When integrating AI tools in healthcare settings, complex interactions between technologies and primary users are not always fully understood or visible. This deficient and ambiguous understanding hampers attempts by healthcare organizations to adopt AI/ML, and it also creates new challenges for researchers to identify opportunities for simplifying adoption and developing best practices for the use of AI-based solutions. Our study fills this gap by documenting the process of designing, building, and maintaining an AI solution called SepsisWatch at Duke University Health System. We conducted 20 interviews with the team of engineers and scientists that led the multi-year effort to build the tool, integrate it into practice, and maintain the solution. This "Algorithm Journey Map" enumerates all social and technical activities throughout the AI solution's procurement, development, integration, and full lifecycle management. In addition to mapping the "who?" and "what?" of the adoption of the AI tool, we also show several 'lessons learned' throughout the algorithm journey maps including modeling assumptions, stakeholder inclusion, and organizational structure. In doing so, we identify generalizable insights about how to recognize and navigate barriers to AI/ML adoption in healthcare settings. We expect that this effort will further the development of best practices for operationalizing and sustaining ethical principles-in algorithmic systems.

APLUS: A Python library for usefulness simulations of machine learning models in healthcare

Despite the creation of thousands of machine learning (ML) models, the promise of improving patient care with ML remains largely unrealized. Adoption into clinical practice is lagging, in large part due to disconnects between how ML practitioners evaluate models and what is required for their successful integration into care delivery. Models are just one component of care delivery workflows whose constraints determine clinicians' abilities to act on models' outputs. However, methods to evaluate the usefulness of models in the context of their corresponding workflows are currently limited. To bridge this gap we developed APLUS, a reusable framework for quantitatively assessing via simulation the utility gained from integrating a model into a clinical workflow. We describe the APLUS simulation engine and workflow specification language, and apply it to evaluate a novel ML-based screening pathway for detecting peripheral artery disease at Stanford Health Care.

Artificial intelligence and clinical deterioration

PURPOSE OF REVIEW

To provide an overview of the systems being used to identify and predict clinical deterioration in hospitalised patients, with focus on the current and future role of artificial intelligence (AI).

RECENT FINDINGS

There are five leading AI driven systems in this field: the Advanced Alert Monitor (AAM), the electronic Cardiac Arrest Risk Triage (eCART) score, Hospital wide Alert Via Electronic Noticeboard, the Mayo Clinic Early Warning Score, and the Rothman Index (RI). Each uses Electronic Patient Record (EPR) data and machine learning to predict adverse events. Less mature but relevant evolutions are occurring in the fields of Natural Language Processing, Time and Motion Studies, AI Sepsis and COVID-19 algorithms.

SUMMARY

Research-based AI-driven systems to predict clinical deterioration are increasingly being developed, but few are being implemented into clinical workflows. Escobar et al. (AAM) provide the current gold standard for robust model development and implementation methodology. Multiple technologies show promise, however, the pathway to meaningfully affect patient outcomes remains challenging.

Situation awareness and the mitigation of risk associated with patient deterioration: A meta-narrative review of theories and models and their relevance to nursing practice

BACKGROUND

Accurate situation awareness has been identified as a critical component of effective deteriorating patient response systems and an essential patient safety skill for nursing practice. However, situation awareness has been defined and theorised from multiple perspectives to explain how individuals, teams and systems maintain awareness in dynamic task environments.

AIM

Our aim was to critically analyse the different approaches taken to the study of situation awareness in healthcare and explore the implications for nursing practice and research as it relates to clinical deterioration in ward contexts.

METHODS

We undertook a meta-narrative review of the healthcare literature to capture how situation awareness has been defined, theorised and studied in healthcare. Following an initial scoping review, we conducted an extensive search of ten electronic databases and included any theoretical, empirical or critical papers with a primary focus on situation awareness in an inpatient hospital setting. Included papers were collaboratively categorised in accordance with their theoretical framing, research tradition and paradigm with a narrative review presented.

RESULTS

A total of 120 papers were included in this review. Three overarching narratives reflecting philosophical, patient safety and solution focussed framings of situation awareness and seven meta-narratives were identified as follows: individual, team and systems perspectives of situation awareness (meta-narratives 1-3), situation awareness and patient safety (meta-narrative 4), communication tools, technologies and education to support situation awareness (meta-narratives 5-7). We identified a concentration of literature from anaesthesia and operating rooms and a body of research largely located within a cognitive engineering tradition and a positivist research paradigm. Endsley's situation awareness model was applied in over 80% of the papers reviewed. A minority of papers drew on alternative situation awareness theories including constructivist, collaborative and distributed perspectives.

CONCLUSIONS

Nurses have a critical role in identifying and escalating the care of deteriorating patients. There is a need to build on prior studies and reflect on the reality of nurse's work and the constraints imposed on situation awareness by the demands of busy inpatient wards. We suggest that this will require an analysis that complements but goes beyond the dominant cognitive engineering tradition to reflect the complex socio-cultural reality of ward-based teams and to explore how situation awareness emerges in increasingly complex, technologically enabled distributed healthcare systems.

Predicting Patient Deterioration: A Review of Tools in the Digital Hospital Setting

BACKGROUND

Early warning tools identify patients at risk of deterioration in hospitals. Electronic medical records in hospitals offer real-time data and the opportunity to automate early warning tools and provide real-time, dynamic risk estimates.

OBJECTIVE

This review describes published studies on the development, validation, and implementation of tools for predicting patient deterioration in general wards in hospitals.

METHODS

An electronic database search of peer reviewed journal papers from 2008-2020 identified studies reporting the use of tools and algorithms for predicting patient deterioration, defined by unplanned transfer to the intensive care unit, cardiac arrest, or death. Studies conducted solely in intensive care units, emergency departments, or single diagnosis patient groups were excluded.

RESULTS

A total of 46 publications were eligible for inclusion. These publications were heterogeneous in design, setting, and outcome measures. Most studies were retrospective studies using cohort data to develop, validate, or statistically evaluate prediction tools. The tools consisted of early warning, screening, or scoring systems based on physiologic data, as well as more complex algorithms developed to better represent real-time data, deal with complexities of longitudinal data, and warn of deterioration risk earlier. Only a few studies detailed the results of the implementation of deterioration warning tools.

CONCLUSIONS

Despite relative progress in the development of algorithms to predict patient deterioration, the literature has not shown that the deployment or implementation of such algorithms is reproducibly associated with improvements in patient outcomes. Further work is needed to realize the potential of automated predictions and update dynamic risk estimates as part of an operational early warning system for inpatient deterioration.

Detecting Deteriorating Patients in the Hospital: Development and Validation of a Novel Scoring System

Rationale: Late recognition of patient deterioration in hospital is associated with worse outcomes, including higher mortality. Despite the widespread introduction of early warning score (EWS) systems and electronic health records, deterioration still goes unrecognized. Objectives: To develop and externally validate a Hospital- wide Alerting via Electronic Noticeboard (HAVEN) system to identify hospitalized patients at risk of reversible deterioration. Methods: This was a retrospective cohort study of patients 16 years of age or above admitted to four UK hospitals. The primary outcome was cardiac arrest or unplanned admission to the ICU. We used patient data (vital signs, laboratory tests, comorbidities, and frailty) from one hospital to train a machine-learning model (gradient boosting trees). We internally and externally validated the model and compared its performance with existing scoring systems (including the National EWS, laboratory-based acute physiology score, and electronic cardiac arrest risk triage score). Measurements and Main Results: We developed the HAVEN model using 230,415 patient admissions to a single hospital. We validated HAVEN on 266,295 admissions to four hospitals. HAVEN showed substantially higher discrimination (c-statistic, 0.901 [95% confidence interval, 0.898-0.903]) for the primary outcome within 24 hours of each measurement than other published scoring systems (which range from 0.700 [0.696-0.704] to 0.863 [0.860-0.865]). With a precision of 10%, HAVEN was able to identify 42% of cardiac arrests or unplanned ICU admissions with a lead time of up to 48 hours in advance, compared with 22% by the next best system. Conclusions: The HAVEN machine-learning algorithm for early identification of in-hospital deterioration significantly outperforms other published scores such as the National EWS.

A framework for making predictive models useful in practice

OBJECTIVE

To analyze the impact of factors in healthcare delivery on the net benefit of triggering an Advanced Care Planning (ACP) workflow based on predictions of 12-month mortality.

MATERIALS AND METHODS

We built a predictive model of 12-month mortality using electronic health record data and evaluated the impact of healthcare delivery factors on the net benefit of triggering an ACP workflow based on the models' predictions. Factors included nonclinical reasons that make ACP inappropriate: limited capacity for ACP, inability to follow up due to patient discharge, and availability of an outpatient workflow to follow up on missed cases. We also quantified the relative benefits of increasing capacity for inpatient ACP versus outpatient ACP.

RESULTS

Work capacity constraints and discharge timing can significantly reduce the net benefit of triggering the ACP workflow based on a model's predictions. However, the reduction can be mitigated by creating an outpatient ACP workflow. Given limited resources to either add capacity for inpatient ACP versus developing outpatient ACP capability, the latter is likely to provide more benefit to patient care.

DISCUSSION

The benefit of using a predictive model for identifying patients for interventions is highly dependent on the capacity to execute the workflow triggered by the model. We provide a framework for quantifying the impact of healthcare delivery factors and work capacity constraints on achieved benefit.

CONCLUSION

An analysis of the sensitivity of the net benefit realized by a predictive model triggered clinical workflow to various healthcare delivery factors is necessary for making predictive models useful in practice.

Assessing clinical heterogeneity in sepsis through treatment patterns and machine learning

OBJECTIVE

To use unsupervised topic modeling to evaluate heterogeneity in sepsis treatment patterns contained within granular data of electronic health records.

MATERIALS AND METHODS

A multicenter, retrospective cohort study of 29 253 hospitalized adult sepsis patients between 2010 and 2013 in Northern California. We applied an unsupervised machine learning method, Latent Dirichlet Allocation, to the orders, medications, and procedures recorded in the electronic health record within the first 24 hours of each patient's hospitalization to uncover empiric treatment topics across the cohort and to develop computable clinical signatures for each patient based on proportions of these topics. We evaluated how these topics correlated with common sepsis treatment and outcome metrics including inpatient mortality, time to first antibiotic, and fluids given within 24 hours.

RESULTS

Mean age was 70 ± 17 years with hospital mortality of 9.6%. We empirically identified 42 clinically recognizable treatment topics (eg, pneumonia, cellulitis, wound care, shock). Only 43.1% of hospitalizations had a single dominant topic, and a small minority (7.3%) had a single topic comprising at least 80% of their overall clinical signature. Across the entire sepsis cohort, clinical signatures were highly variable.

DISCUSSION

Heterogeneity in sepsis is a major barrier to improving targeted treatments, yet existing approaches to characterizing clinical heterogeneity are narrowly defined. A machine learning approach captured substantial patient- and population-level heterogeneity in treatment during early sepsis hospitalization.

CONCLUSION

Using topic modeling based on treatment patterns may enable more precise clinical characterization in sepsis and better understanding of variability in sepsis presentation and outcomes.

Prediction of obstetrical and fetal complications using automated electronic health record data

An increasing number of delivering women experience major morbidity and mortality. Limited work has been done on automated predictive models that could be used for prevention. Using only routinely collected obstetrical data, this study aimed to develop a predictive model suitable for real-time use with an electronic medical record. We used a retrospective cohort study design with split validation. The denominator consisted of women admitted to a delivery service. The numerator consisted of women who experienced a composite outcome that included both maternal (eg, uterine rupture, postpartum hemorrhage), fetal (eg, stillbirth), and neonatal (eg, hypoxic ischemic encephalopathy) adverse events. We employed machine learning methods, assessing model performance using the area under the receiver operator characteristic curve and number needed to evaluate. A total of 303,678 deliveries took place at 15 study hospitals between January 1, 2010, and March 31, 2018, and 4130 (1.36%) had ≥1 obstetrical complication. We employed data from 209,611 randomly selected deliveries (January 1, 2010, to March 31, 2017) as a derivation dataset and validated our findings on data from 52,398 randomly selected deliveries during the same time period (validation 1 dataset). We then applied our model to data from 41,669 deliveries from the last year of the study (April 1, 2017, to March 31, 2018 [validation 2 dataset]). Our model included 35 variables (eg, demographics, vital signs, laboratory tests, progress of labor indicators). In the validation 2 dataset, a gradient boosted model (area under the receiver operating characteristic curve or c statistic, 0.786) was slightly superior to a logistic regression model (c statistic, 0.778). Using an alert threshold of 4.1%, our final model would flag 16.7% of women and detect 52% of adverse outcomes, with a number needed to evaluate of 20.9 and 0.455 first alerts per day per 1000 annual deliveries. In conclusion, electronic medical record data can be used to predict obstetrical complications. The clinical utility of these automated models has not yet been demonstrated. To conduct interventions to assess whether using these models results in patient benefit, future work will need to focus on the development of clinical protocols suitable for use in interventions.

Automated Identification of Adults at Risk for In-Hospital Clinical Deterioration

BACKGROUND

Hospitalized adults whose condition deteriorates while they are in wards (outside the intensive care unit [ICU]) have considerable morbidity and mortality. Early identification of patients at risk for clinical deterioration has relied on manually calculated scores. Outcomes after an automated detection of impending clinical deterioration have not been widely reported.

METHODS

On the basis of a validated model that uses information from electronic medical records to identify hospitalized patients at high risk for clinical deterioration (which permits automated, real-time risk-score calculation), we developed an intervention program involving remote monitoring by nurses who reviewed records of patients who had been identified as being at high risk; results of this monitoring were then communicated to rapid-response teams at hospitals. We compared outcomes (including the primary outcome, mortality within 30 days after an alert) among hospitalized patients (excluding those in the ICU) whose condition reached the alert threshold at hospitals where the system was operational (intervention sites, where alerts led to a clinical response) with outcomes among patients at hospitals where the system had not yet been deployed (comparison sites, where a patient's condition would have triggered a clinical response after an alert had the system been operational). Multivariate analyses adjusted for demographic characteristics, severity of illness, and burden of coexisting conditions.

RESULTS

The program was deployed in a staggered fashion at 19 hospitals between August 1, 2016, and February 28, 2019. We identified 548,838 non-ICU hospitalizations involving 326,816 patients. A total of 43,949 hospitalizations (involving 35,669 patients) involved a patient whose condition reached the alert threshold; 15,487 hospitalizations were included in the intervention cohort, and 28,462 hospitalizations in the comparison cohort. Mortality within 30 days after an alert was lower in the intervention cohort than in the comparison cohort (adjusted relative risk, 0.84, 95% confidence interval, 0.78 to 0.90; P<0.001).

CONCLUSIONS

The use of an automated predictive model to identify high-risk patients for whom interventions by rapid-response teams could be implemented was associated with decreased mortality. (Funded by the Gordon and Betty Moore Foundation and others.).

What Do We Do After the Pilot Is Done? Implementation of a Hospital Early Warning System at Scale

BACKGROUND

Adults who deteriorate outside the ICU have high mortality. Most rapid response systems (RRSs) have employed manual detection processes that rapid response teams (RRTs) use to identify patients at risk. This project piloted the use of an automated early warning system (EWS), based on a very large database, that provides RRTs with 12 hours lead time to mount a response. Results from a 2-hospital pilot were encouraging, so leadership decided to deploy the Advance Alert Monitor (AAM) program in 19 more hospitals.

CHALLENGE

How can one deploy an RRS using an automated EWS at scale?

SOLUTION

EWS displays were removed from frontline clinicians' hospital electronic dashboards, and a Virtual Quality Team (VQT) RN was interposed between the EWS and the RRT. VQT RNs monitor the EWS remotely-when alerts are issued, they conduct a preliminary chart review and contact hospital RRT RNs. VQT and RRT RNs review the cases jointly. The RRT RNs then consult with hospitalists regarding clinical rescue and/or palliative care workflows. Subsequently, VQT RNs monitor patient charts, ensuring adherence to RRS practice standards. To enable this process, the project team developed a governance structure, clinical workflows, palliative care workflows, and documentation standards.

RESULTS

The AAM Program now functions in 21 Kaiser Permanente Northern California hospitals. VQT RNs monitor EWS alerts 24 hours a day, 7 days a week. The AAM Program handles ∼16,000 alerts per year. Its implementation has resulted in standardization of RRT staffing, clinical rescue workflows, and in-hospital palliative care.

The number needed to benefit: estimating the value of predictive analytics in healthcare

Predictive analytics in health care has generated increasing enthusiasm recently, as reflected in a rapidly growing body of predictive models reported in literature and in real-time embedded models using electronic health record data. However, estimating the benefit of applying any single model to a specific clinical problem remains challenging today. Developing a shared framework for estimating model value is therefore critical to facilitate the effective, safe, and sustainable use of predictive tools into the future. We highlight key concepts within the prediction-action dyad that together are expected to impact model benefit. These include factors relevant to model prediction (including the number needed to screen) as well as those relevant to the subsequent action (number needed to treat). In the simplest terms, a number needed to benefit contextualizes the numbers needed to screen and treat, offering an opportunity to estimate the value of a clinical predictive model in action.

Early Warning Score Communication Bundle: A Pilot Study

BACKGROUND

Unplanned admissions of patients to intensive care units from medical-surgical units often result from failure to recognize clinical deterioration. The early warning score is a clinical decision support tool for nurse surveillance but must be communicated to nurses and implemented appropriately. A communication process including collaboration with experienced intensive care unit nurses may reduce unplanned transfers.

OBJECTIVE

To determine the impact of an early warning score communication bundle on medical-surgical transfers to the intensive care unit, rapid response team calls, and morbidity of patients upon intensive care unit transfer.

METHODS

After an early warning score was electronically embedded into medical records, a communication bundle including notification of and telephone collaboration between medical-surgical and intensive care unit nurses was implemented. Data were collected 3 months before and 21 months after implementation.

RESULTS

Rapid response team calls increased nonsignificantly during the study period (from 6.47 to 8.29 per 1000 patient-days). Rapid response team calls for patients with early warning scores greater than 4 declined (from 2.04 to 1.77 per 1000 patient-days). Intensive care unit admissions of patients after rapid response team calls significantly declined (P = .03), as did admissions of patients with early warning scores greater than 4 (P = .01), suggesting that earlier intervention for patient deterioration occurred. Documented reassessment response time declined significantly to 28 minutes (P = .002).

CONCLUSION

Electronic surveillance and collaboration with experienced intensive care unit nurses may improve care, control costs, and save lives. Critical care nurses have a role in coaching and guiding less experienced nurses.

Automated early detection of obstetric complications: theoretic and methodologic considerations

Compared with adults who are admitted to general medical-surgical wards, women who are admitted to labor and delivery services are at much lower risk of experiencing unexpected critical illness. Nonetheless, critical illness and other complications that put either the mother or fetus at risk do occur. One potential approach to prevention is to use automated early warning systems, such as those used for nonpregnant adults. Predictive models that use data extracted in real time from electronic records constitute the cornerstone of such systems. This article addresses several issues that are involved in the development of such predictive models: specification of temporal characteristics, choice of denominator, selection of outcomes for model calibration, potential uses of existing adult severity of illness scores, approaches to data processing, statistical considerations, validation, and options for instantiation. These have not been addressed explicitly in the obstetrics literature, which has focused on the use of manually assigned scores. In addition, this article provides some results from work in progress to develop 2 obstetric predictive models with the use of data from 262,071 women who were admitted to a labor and delivery service at 15 Kaiser Permanente Northern California hospitals between 2010 and 2017.

[Consensus document for sepsis code implementation and development in the Community of Madrid]

The consensus paper for the implementation and development of the sepsis code, finished in April 2017 is presented here. It was adopted by the Regional Office of Health as a working document for the implementation of the sepsis code in the Community of Madrid, both in the hospital setting (acute, middle and long-stay hospitals) and in Primary Care and Out-of-Hospital Emergency Services. It is now published without changes with respect to the original version, having only added the most significant bibliographical references. The document is divided into four parts: introduction, initial detection and assessment, early therapy and organizational recommendations. In the second to fourth sections, 25 statements or proposals have been included, agreed upon by the authors after several face-to-face meetings and an extensive "online" discussion. The annex includes nine tables that are intended as a practical guide to the activation of the sepsis code. Both the content of the recommendations and their formal writing have been made taking into account their applicability in all areas to which they are directed, which may have very different structural and functional characteristics and features, so that we have deliberately avoided a greater degree of concretion: the objective is not that the sepsis code is organized and applied identically in all of them, but that the health resources work in a coordinated manner aligned in the same direction.

Predictive Modeling of 30-Day Emergency Hospital Transport of Patients Using a Personal Emergency Response System: Prognostic Retrospective Study

BACKGROUND

Telehealth programs have been successful in reducing 30-day readmissions and emergency department visits. However, such programs often focus on the costliest patients with multiple morbidities and last for only 30 to 60 days postdischarge. Inexpensive monitoring of elderly patients via a personal emergency response system (PERS) to identify those at high risk for emergency hospital transport could be used to target interventions and prevent avoidable use of costly readmissions and emergency department visits after 30 to 60 days of telehealth use.

OBJECTIVE

The objectives of this study were to (1) develop and validate a predictive model of 30-day emergency hospital transport based on PERS data; and (2) compare the model's predictions with clinical outcomes derived from the electronic health record (EHR).

METHODS

We used deidentified medical alert pattern data from 290,434 subscribers to a PERS service to build a gradient tree boosting-based predictive model of 30-day hospital transport, which included predictors derived from subscriber demographics, self-reported medical conditions, caregiver network information, and up to 2 years of retrospective PERS medical alert data. We evaluated the model's performance on an independent validation cohort (n=289,426). We linked EHR and PERS records for 1815 patients from a home health care program to compare PERS-based risk scores with rates of emergency encounters as recorded in the EHR.

RESULTS

In the validation cohort, 2.22% (6411/289,426) of patients had 1 or more emergency transports in 30 days. The performance of the predictive model of emergency hospital transport, as evaluated by the area under the receiver operating characteristic curve, was 0.779 (95% CI 0.774-0.785). Among the top 1% of predicted high-risk patients, 25.5% had 1 or more emergency hospital transports in the next 30 days. Comparison with clinical outcomes from the EHR showed 3.9 times more emergency encounters among predicted high-risk patients than low-risk patients in the year following the prediction date.

CONCLUSIONS

Patient data collected remotely via PERS can be used to reliably predict 30-day emergency hospital transport. Clinical observations from the EHR showed that predicted high-risk patients had nearly four times higher rates of emergency encounters than did low-risk patients. Health care providers could benefit from our validated predictive model by targeting timely preventive interventions to high-risk patients. This could lead to overall improved patient experience, higher quality of care, and more efficient resource utilization.

The impact of delayed critical care outreach team activation on in-hospital mortality and other patient outcomes: a historical cohort study

PURPOSE

Early warning scores (EWS) and critical care outreach teams (CCOT) have been developed to respond to decompensating patients. Nevertheless, controversy exists around their effectiveness. The primary objective of this study was to determine if a delay of ≥ 60 min between when a patient was identified as meeting EWS criteria and the CCOT was activated impacted in-hospital mortality.

METHODS

This was a historical cohort study evaluating all new CCOT activations over a four-year study period (1 June 2007 to 31 August 2011) for inpatients ≥ 18 yr of age at two academic tertiary care hospitals in London, Ontario, Canada. Multivariable logistic regression accounting for repeated measures was used to determine the effect of delayed CCOT activation on in-hospital mortality (primary outcome). Differences in outcomes between medical and surgical patients were also examined.

RESULTS

There were 3,133 CCOT activations for 1,684 (53.8%) medical patients and 1,449 (46.2%) surgical patients during the study period. The CCOT was activated < 60 min of a patient meeting EWS criteria in 2,160 (68.9%) cases and ≥ 60 min in 973 (31.1%) cases. Patients with ≥ 60 min delay were more likely be admitted to the intensive care unit (odds ratio [OR], 1.22; 95% confidence interval [CI], 1.07 to 1.47) and to suffer in-hospital mortality (OR, 1.30; 95% CI, 1.08 to 1.56). Irrespective of delay, surgical patients were less likely to experience in-hospital mortality than medical patients (OR, 0.46; 95% CI, 0.39 to 0.55).

CONCLUSION

Including the rates of delay in CCOT activation and the admitting service could be an additional step in exploring the conflicting results seen in the current literature assessing the impact of CCOT on patient outcomes.

Early detection of critical illness outside the intensive care unit: Clarifying treatment plans and honoring goals of care using a supportive care team

Given the high mortality experienced by patients who deteriorate outside the intensive care unit, issues related to patient preferences around escalation of care are common. However, the literature on early warning systems (EWSs) provides limited information on how respecting patient preferences can be incorporated into clinical workflows. In this report, we describe how we developed workflows for integrating supportive care with an automated EWS in the context of a 2-hospital pilot. We used the Institute for Healthcare Improvement's Plan-Do-Study-Act approach to achieve consensus with clinicians and administrators. The workflows will serve as the basis for dissemination to an additional 19 hospitals. We were successful in integrating an automated EWS with supportive care. Our workflows take local resource availability into account and have been well received by hospitalists, nurses, and families. Our work demonstrates that one can achieve integration of proactive supportive care into the operation of an EWS. Creation of a palliative care response arm that is complementary to a clinical rescue arm ensures that patient preferences are respected. Journal of Hospital Medicine 2016;11:S40-S47. © 2016 Society of Hospital Medicine.

Piloting electronic medical record-based early detection of inpatient deterioration in community hospitals

Patients who deteriorate in the hospital outside the intensive care unit (ICU) have higher mortality and morbidity than those admitted directly to the ICU. As more hospitals deploy comprehensive inpatient electronic medical records (EMRs), attempts to support rapid response teams with automated early detection systems are becoming more frequent. We aimed to describe some of the technical and operational challenges involved in the deployment of an early detection system. This 2-hospital pilot, set within an integrated healthcare delivery system with 21 hospitals, had 2 objectives. First, it aimed to demonstrate that severity scores and probability estimates could be provided to hospitalists in real time. Second, it aimed to surface issues that would need to be addressed so that deployment of the early warning system could occur in all remaining hospitals. To achieve these objectives, we first established a rationale for the development of an early detection system through the analysis of risk-adjusted outcomes. We then demonstrated that EMR data could be employed to predict deteriorations. After addressing specific organizational mandates (eg, defining the clinical response to a probability estimate), we instantiated a set of equations into a Java application that transmits scores and probability estimates so that they are visible in a commercially available EMR every 6 hours. The pilot has been successful and deployment to the remaining hospitals has begun. Journal of Hospital Medicine 2016;11:S18-S24. © 2016 Society of Hospital Medicine.