1
|
Tupper HI, Lawson BL, Kipnis P, Patel AR, Ashiku SK, Roubinian NH, Myers LC, Liu VX, Velotta JB. Video-Assisted vs Robotic-Assisted Lung Lobectomies for Operating Room Resource Utilization and Patient Outcomes. JAMA Netw Open 2024; 7:e248881. [PMID: 38700865 PMCID: PMC11069083 DOI: 10.1001/jamanetworkopen.2024.8881] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/07/2023] [Accepted: 02/09/2024] [Indexed: 05/06/2024] Open
Abstract
Importance With increased use of robots, there is an inadequate understanding of minimally invasive modalities' time costs. This study evaluates the operative durations of robotic-assisted vs video-assisted lung lobectomies. Objective To compare resource utilization, specifically operative time, between video-assisted and robotic-assisted thoracoscopic lung lobectomies. Design, Setting, and Participants This retrospective cohort study evaluated patients aged 18 to 90 years who underwent minimally invasive (robotic-assisted or video-assisted) lung lobectomy from January 1, 2020, to December 31, 2022, with 90 days' follow-up after surgery. The study included multicenter electronic health record data from 21 hospitals within an integrated health care system in Northern California. Thoracic surgery was regionalized to 4 centers with 14 board-certified general thoracic surgeons. Exposures Robotic-assisted or video-assisted lung lobectomy. Main Outcomes and Measures The primary outcome was operative duration (cut to close) in minutes. Secondary outcomes were length of stay, 30-day readmission, and 90-day mortality. Comparisons between video-assisted and robotic-assisted lobectomies were generated using the Wilcoxon rank sum test for continuous variables and the χ2 test for categorical variables. The average treatment effects were estimated with augmented inverse probability treatment weighting (AIPTW). Patient and surgeon covariates were adjusted for and included patient demographics, comorbidities, and case complexity (age, sex, race and ethnicity, neighborhood deprivation index, body mass index, Charlson Comorbidity Index score, nonelective hospitalizations, emergency department visits, a validated laboratory derangement score, a validated institutional comorbidity score, a surgeon-designated complexity indicator, and a procedural code count), and a primary surgeon-specific indicator. Results The study included 1088 patients (median age, 70.1 years [IQR, 63.3-75.8 years]; 704 [64.7%] female), of whom 446 (41.0%) underwent robotic-assisted and 642 (59.0%) underwent video-assisted lobectomy. The median unadjusted operative duration was 172.0 minutes (IQR, 128.0-226.0 minutes). After AIPTW, there was less than a 10% difference in all covariates between groups, and operative duration was a median 20.6 minutes (95% CI, 12.9-28.2 minutes; P < .001) longer for robotic-assisted compared with video-assisted lobectomies. There was no difference in adjusted secondary patient outcomes, specifically for length of stay (0.3 days; 95% CI, -0.3 to 0.8 days; P = .11) or risk of 30-day readmission (adjusted odds ratio, 1.29; 95% CI, 0.84-1.98; P = .13). The unadjusted 90-day mortality rate (1.3% [n = 14]) was too low for the AIPTW modeling process. Conclusions and Relevance In this cohort study, there was no difference in patient outcomes between modalities, but operative duration was longer in robotic-assisted compared with video-assisted lung lobectomy. Given that this elevated operative duration is additive when applied systematically, increased consideration of appropriate patient selection for robotic-assisted lung lobectomy is needed to improve resource utilization.
Collapse
Affiliation(s)
- Haley I. Tupper
- Division of General Surgery, Department of Surgery, University of California, Los Angeles
| | - Brian L. Lawson
- Division of Research, Kaiser Permanente Northern California, Oakland
| | - Patricia Kipnis
- Division of Research, Kaiser Permanente Northern California, Oakland
| | - Ashish R. Patel
- Division of Thoracic Surgery, Department of Surgery, Kaiser Permanente Oakland, Oakland, California
| | - Simon K. Ashiku
- Division of Thoracic Surgery, Department of Surgery, Kaiser Permanente Oakland, Oakland, California
| | - Nareg H. Roubinian
- Division of Research, Kaiser Permanente Northern California, Oakland
- Kaiser Permanente Bernard J. Tyson School of Medicine, Pasadena, California
| | - Laura C. Myers
- Division of Research, Kaiser Permanente Northern California, Oakland
| | - Vincent X. Liu
- Division of Research, Kaiser Permanente Northern California, Oakland
| | - Jeffrey B. Velotta
- Division of Thoracic Surgery, Department of Surgery, Kaiser Permanente Oakland, Oakland, California
- Kaiser Permanente Bernard J. Tyson School of Medicine, Pasadena, California
- Department of Surgery, University of California San Francisco School of Medicine
| |
Collapse
|
2
|
Boag W, Hasan A, Kim JY, Revoir M, Nichols M, Ratliff W, Gao M, Zilberstein S, Samad Z, Hoodbhoy Z, Ali M, Khan NS, Patel M, Balu S, Sendak M. The algorithm journey map: a tangible approach to implementing AI solutions in healthcare. NPJ Digit Med 2024; 7:87. [PMID: 38594344 PMCID: PMC11003994 DOI: 10.1038/s41746-024-01061-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2023] [Accepted: 02/19/2024] [Indexed: 04/11/2024] Open
Abstract
When integrating AI tools in healthcare settings, complex interactions between technologies and primary users are not always fully understood or visible. This deficient and ambiguous understanding hampers attempts by healthcare organizations to adopt AI/ML, and it also creates new challenges for researchers to identify opportunities for simplifying adoption and developing best practices for the use of AI-based solutions. Our study fills this gap by documenting the process of designing, building, and maintaining an AI solution called SepsisWatch at Duke University Health System. We conducted 20 interviews with the team of engineers and scientists that led the multi-year effort to build the tool, integrate it into practice, and maintain the solution. This "Algorithm Journey Map" enumerates all social and technical activities throughout the AI solution's procurement, development, integration, and full lifecycle management. In addition to mapping the "who?" and "what?" of the adoption of the AI tool, we also show several 'lessons learned' throughout the algorithm journey maps including modeling assumptions, stakeholder inclusion, and organizational structure. In doing so, we identify generalizable insights about how to recognize and navigate barriers to AI/ML adoption in healthcare settings. We expect that this effort will further the development of best practices for operationalizing and sustaining ethical principles-in algorithmic systems.
Collapse
Affiliation(s)
- William Boag
- Duke Institute for Health Innovation, Durham, NC, USA
| | - Alifia Hasan
- Duke Institute for Health Innovation, Durham, NC, USA
| | - Jee Young Kim
- Duke Institute for Health Innovation, Durham, NC, USA
| | - Mike Revoir
- Duke Institute for Health Innovation, Durham, NC, USA
| | | | | | - Michael Gao
- Duke Institute for Health Innovation, Durham, NC, USA
| | - Shira Zilberstein
- Duke Institute for Health Innovation, Durham, NC, USA
- Harvard University, Cambridge, MA, USA
| | | | | | | | | | - Manesh Patel
- Duke University School of Medicine, Durham, NC, USA
| | - Suresh Balu
- Duke Institute for Health Innovation, Durham, NC, USA
| | - Mark Sendak
- Duke Institute for Health Innovation, Durham, NC, USA.
| |
Collapse
|
3
|
APLUS: A Python library for usefulness simulations of machine learning models in healthcare. J Biomed Inform 2023; 139:104319. [PMID: 36791900 DOI: 10.1016/j.jbi.2023.104319] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2022] [Revised: 02/09/2023] [Accepted: 02/10/2023] [Indexed: 02/16/2023]
Abstract
Despite the creation of thousands of machine learning (ML) models, the promise of improving patient care with ML remains largely unrealized. Adoption into clinical practice is lagging, in large part due to disconnects between how ML practitioners evaluate models and what is required for their successful integration into care delivery. Models are just one component of care delivery workflows whose constraints determine clinicians' abilities to act on models' outputs. However, methods to evaluate the usefulness of models in the context of their corresponding workflows are currently limited. To bridge this gap we developed APLUS, a reusable framework for quantitatively assessing via simulation the utility gained from integrating a model into a clinical workflow. We describe the APLUS simulation engine and workflow specification language, and apply it to evaluate a novel ML-based screening pathway for detecting peripheral artery disease at Stanford Health Care.
Collapse
|
4
|
Abstract
PURPOSE OF REVIEW To provide an overview of the systems being used to identify and predict clinical deterioration in hospitalised patients, with focus on the current and future role of artificial intelligence (AI). RECENT FINDINGS There are five leading AI driven systems in this field: the Advanced Alert Monitor (AAM), the electronic Cardiac Arrest Risk Triage (eCART) score, Hospital wide Alert Via Electronic Noticeboard, the Mayo Clinic Early Warning Score, and the Rothman Index (RI). Each uses Electronic Patient Record (EPR) data and machine learning to predict adverse events. Less mature but relevant evolutions are occurring in the fields of Natural Language Processing, Time and Motion Studies, AI Sepsis and COVID-19 algorithms. SUMMARY Research-based AI-driven systems to predict clinical deterioration are increasingly being developed, but few are being implemented into clinical workflows. Escobar et al. (AAM) provide the current gold standard for robust model development and implementation methodology. Multiple technologies show promise, however, the pathway to meaningfully affect patient outcomes remains challenging.
Collapse
Affiliation(s)
- James Malycha
- Discipline of Acute Care Medicine, University of Adelaide, Adelaide
- The Queen Elizabeth Hospital, Department of Intensive Care Medicine, Woodville South
| | - Stephen Bacchi
- Royal Adelaide Hospital, Adelaide, South Australia, Australia
| | - Oliver Redfern
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
| |
Collapse
|
5
|
Walshe N, Ryng S, Drennan J, O'Connor P, O'Brien S, Crowley C, Hegarty J. Situation awareness and the mitigation of risk associated with patient deterioration: A meta-narrative review of theories and models and their relevance to nursing practice. Int J Nurs Stud 2021; 124:104086. [PMID: 34601204 DOI: 10.1016/j.ijnurstu.2021.104086] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2021] [Revised: 07/27/2021] [Accepted: 08/31/2021] [Indexed: 10/20/2022]
Abstract
BACKGROUND Accurate situation awareness has been identified as a critical component of effective deteriorating patient response systems and an essential patient safety skill for nursing practice. However, situation awareness has been defined and theorised from multiple perspectives to explain how individuals, teams and systems maintain awareness in dynamic task environments. AIM Our aim was to critically analyse the different approaches taken to the study of situation awareness in healthcare and explore the implications for nursing practice and research as it relates to clinical deterioration in ward contexts. METHODS We undertook a meta-narrative review of the healthcare literature to capture how situation awareness has been defined, theorised and studied in healthcare. Following an initial scoping review, we conducted an extensive search of ten electronic databases and included any theoretical, empirical or critical papers with a primary focus on situation awareness in an inpatient hospital setting. Included papers were collaboratively categorised in accordance with their theoretical framing, research tradition and paradigm with a narrative review presented. RESULTS A total of 120 papers were included in this review. Three overarching narratives reflecting philosophical, patient safety and solution focussed framings of situation awareness and seven meta-narratives were identified as follows: individual, team and systems perspectives of situation awareness (meta-narratives 1-3), situation awareness and patient safety (meta-narrative 4), communication tools, technologies and education to support situation awareness (meta-narratives 5-7). We identified a concentration of literature from anaesthesia and operating rooms and a body of research largely located within a cognitive engineering tradition and a positivist research paradigm. Endsley's situation awareness model was applied in over 80% of the papers reviewed. A minority of papers drew on alternative situation awareness theories including constructivist, collaborative and distributed perspectives. CONCLUSIONS Nurses have a critical role in identifying and escalating the care of deteriorating patients. There is a need to build on prior studies and reflect on the reality of nurse's work and the constraints imposed on situation awareness by the demands of busy inpatient wards. We suggest that this will require an analysis that complements but goes beyond the dominant cognitive engineering tradition to reflect the complex socio-cultural reality of ward-based teams and to explore how situation awareness emerges in increasingly complex, technologically enabled distributed healthcare systems.
Collapse
Affiliation(s)
- Nuala Walshe
- School of Nursing and Midwifery, University College Cork, College Road, Cork T12 AK54, Ireland.
| | - Stephanie Ryng
- School of Nursing and Midwifery, University College Cork, College Road, Cork T12 AK54, Ireland
| | - Jonathan Drennan
- School of Nursing and Midwifery, University College Cork, College Road, Cork T12 AK54, Ireland.
| | - Paul O'Connor
- Department of General Practice, National University of Ireland, Distillery Road, Newcastle, Co Galway H91 TK33, Ireland.
| | - Sinéad O'Brien
- School of Nursing and Midwifery, University College Cork, College Road, Cork T12 AK54, Ireland.
| | - Clare Crowley
- School of Nursing and Midwifery, University College Cork, College Road, Cork T12 AK54, Ireland.
| | - Josephine Hegarty
- School of Nursing and Midwifery, University College Cork, College Road, Cork T12 AK54, Ireland.
| |
Collapse
|
6
|
Mann KD, Good NM, Fatehi F, Khanna S, Campbell V, Conway R, Sullivan C, Staib A, Joyce C, Cook D. Predicting Patient Deterioration: A Review of Tools in the Digital Hospital Setting. J Med Internet Res 2021; 23:e28209. [PMID: 34591017 PMCID: PMC8517822 DOI: 10.2196/28209] [Citation(s) in RCA: 21] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2021] [Revised: 07/14/2021] [Accepted: 07/27/2021] [Indexed: 12/23/2022] Open
Abstract
BACKGROUND Early warning tools identify patients at risk of deterioration in hospitals. Electronic medical records in hospitals offer real-time data and the opportunity to automate early warning tools and provide real-time, dynamic risk estimates. OBJECTIVE This review describes published studies on the development, validation, and implementation of tools for predicting patient deterioration in general wards in hospitals. METHODS An electronic database search of peer reviewed journal papers from 2008-2020 identified studies reporting the use of tools and algorithms for predicting patient deterioration, defined by unplanned transfer to the intensive care unit, cardiac arrest, or death. Studies conducted solely in intensive care units, emergency departments, or single diagnosis patient groups were excluded. RESULTS A total of 46 publications were eligible for inclusion. These publications were heterogeneous in design, setting, and outcome measures. Most studies were retrospective studies using cohort data to develop, validate, or statistically evaluate prediction tools. The tools consisted of early warning, screening, or scoring systems based on physiologic data, as well as more complex algorithms developed to better represent real-time data, deal with complexities of longitudinal data, and warn of deterioration risk earlier. Only a few studies detailed the results of the implementation of deterioration warning tools. CONCLUSIONS Despite relative progress in the development of algorithms to predict patient deterioration, the literature has not shown that the deployment or implementation of such algorithms is reproducibly associated with improvements in patient outcomes. Further work is needed to realize the potential of automated predictions and update dynamic risk estimates as part of an operational early warning system for inpatient deterioration.
Collapse
Affiliation(s)
- Kay D Mann
- The Australian e-Health Research Centre, Commonwealth Scientific and Industrial Research Organisation, Brisbane, Australia
| | - Norm M Good
- The Australian e-Health Research Centre, Commonwealth Scientific and Industrial Research Organisation, Brisbane, Australia
| | - Farhad Fatehi
- Centre for Health Services Research, Faculty of Medicine, The University of Queensland, Brisbane, Australia.,School of Psychological Sciences, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia
| | - Sankalp Khanna
- The Australian e-Health Research Centre, Commonwealth Scientific and Industrial Research Organisation, Brisbane, Australia
| | - Victoria Campbell
- Sunshine Coast University Hospital, Sunshine Coast Hospital and Health Service, Birtinya, Australia.,Clinical Excellence Queensland, Queensland Health, Queensland, Australia.,School of Medicine, Griffith University, Nathan Campas, Australia
| | - Roger Conway
- Sunshine Coast University Hospital, Sunshine Coast Hospital and Health Service, Birtinya, Australia
| | - Clair Sullivan
- Centre for Health Services Research, Faculty of Medicine, The University of Queensland, Brisbane, Australia.,Metro North Hospital and Health Service, Brisbane, Australia
| | - Andrew Staib
- Clinical Excellence Queensland, Queensland Health, Queensland, Australia.,Princess Alexandra Hospital, Metro South Hospital and Health Service, Brisbane, Australia.,Faculty of Medicine, The University of Queensland, Brisbane, Australia
| | - Christopher Joyce
- Princess Alexandra Hospital, Metro South Hospital and Health Service, Brisbane, Australia.,Faculty of Medicine, The University of Queensland, Brisbane, Australia
| | - David Cook
- Princess Alexandra Hospital, Metro South Hospital and Health Service, Brisbane, Australia.,Faculty of Medicine, The University of Queensland, Brisbane, Australia.,School of Computer Science, Faculty of Science, Queensland University of Technology, Brisbane, Australia
| |
Collapse
|
7
|
Barr J, Paulson SS, Kamdar B, Ervin JN, Lane-Fall M, Liu V, Kleinpell R. The Coming of Age of Implementation Science and Research in Critical Care Medicine. Crit Care Med 2021; 49:1254-1275. [PMID: 34261925 PMCID: PMC8549627 DOI: 10.1097/ccm.0000000000005131] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/29/2022]
Affiliation(s)
- Juliana Barr
- Anesthesiology and Perioperative Care Service, VA Palo Alto Health Care System, Palo Alto, CA
- Department of Anesthesiology, Perioperative and Pain Medicine, Stanford University School of Medicine, Stanford, CA
| | - Shirley S Paulson
- Regional Adult Patient Care Services, Kaiser Permanente, Northern California, Oakland, CA
| | - Biren Kamdar
- Division of Pulmonary, Critical Care and Sleep Medicine, Department of Medicine, University of California, San Diego School of Medicine, La Jolla, CA
| | - Jennifer N Ervin
- Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, University of Michigan Medical School, Ann Arbor, MI
| | - Meghan Lane-Fall
- Department of Anesthesiology and Critical Care, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
- Penn Implementation Science Center at the Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, PA
| | - Vincent Liu
- Anesthesiology and Perioperative Care Service, VA Palo Alto Health Care System, Palo Alto, CA
- Department of Anesthesiology, Perioperative and Pain Medicine, Stanford University School of Medicine, Stanford, CA
- Regional Adult Patient Care Services, Kaiser Permanente, Northern California, Oakland, CA
- Division of Pulmonary, Critical Care and Sleep Medicine, Department of Medicine, University of California, San Diego School of Medicine, La Jolla, CA
- Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, University of Michigan Medical School, Ann Arbor, MI
- Department of Anesthesiology and Critical Care, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
- Penn Implementation Science Center at the Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, PA
- Division of Research, Kaiser Permanente Northern California, Santa Clara, CA
- Kaiser Permanente Medical Center, Santa Clara, CA
- Stanford University, Stanford, CA
- Hospital Advanced Analytics, Kaiser Permanente Northern California, Santa Clara, CA
- Vanderbilt University School of Nursing, Nashville, TN
| | | |
Collapse
|
8
|
Pimentel MAF, Redfern OC, Malycha J, Meredith P, Prytherch D, Briggs J, Young JD, Clifton DA, Tarassenko L, Watkinson PJ. Detecting Deteriorating Patients in the Hospital: Development and Validation of a Novel Scoring System. Am J Respir Crit Care Med 2021; 204:44-52. [PMID: 33525997 PMCID: PMC8437126 DOI: 10.1164/rccm.202007-2700oc] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2020] [Accepted: 02/01/2021] [Indexed: 12/23/2022] Open
Abstract
Rationale: Late recognition of patient deterioration in hospital is associated with worse outcomes, including higher mortality. Despite the widespread introduction of early warning score (EWS) systems and electronic health records, deterioration still goes unrecognized. Objectives: To develop and externally validate a Hospital- wide Alerting via Electronic Noticeboard (HAVEN) system to identify hospitalized patients at risk of reversible deterioration. Methods: This was a retrospective cohort study of patients 16 years of age or above admitted to four UK hospitals. The primary outcome was cardiac arrest or unplanned admission to the ICU. We used patient data (vital signs, laboratory tests, comorbidities, and frailty) from one hospital to train a machine-learning model (gradient boosting trees). We internally and externally validated the model and compared its performance with existing scoring systems (including the National EWS, laboratory-based acute physiology score, and electronic cardiac arrest risk triage score). Measurements and Main Results: We developed the HAVEN model using 230,415 patient admissions to a single hospital. We validated HAVEN on 266,295 admissions to four hospitals. HAVEN showed substantially higher discrimination (c-statistic, 0.901 [95% confidence interval, 0.898-0.903]) for the primary outcome within 24 hours of each measurement than other published scoring systems (which range from 0.700 [0.696-0.704] to 0.863 [0.860-0.865]). With a precision of 10%, HAVEN was able to identify 42% of cardiac arrests or unplanned ICU admissions with a lead time of up to 48 hours in advance, compared with 22% by the next best system. Conclusions: The HAVEN machine-learning algorithm for early identification of in-hospital deterioration significantly outperforms other published scores such as the National EWS.
Collapse
Affiliation(s)
| | - Oliver C. Redfern
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, United Kingdom
| | - James Malycha
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, United Kingdom
| | - Paul Meredith
- Research and Innovation Department, Portsmouth Hospitals University National Health Service Trust, Portsmouth, United Kingdom
| | - David Prytherch
- Centre for Healthcare Modelling and Informatics, University of Portsmouth, Portsmouth, United Kingdom; and
| | - Jim Briggs
- Centre for Healthcare Modelling and Informatics, University of Portsmouth, Portsmouth, United Kingdom; and
| | - J. Duncan Young
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, United Kingdom
| | - David A. Clifton
- Institute of Biomedical Engineering, Department of Engineering Science, and
| | - Lionel Tarassenko
- Institute of Biomedical Engineering, Department of Engineering Science, and
| | - Peter J. Watkinson
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, United Kingdom
- Kadoorie Centre for Critical Care Research and Education, Oxford University Hospitals National Health Service Trust, Oxford, United Kingdom
| |
Collapse
|
9
|
Jung K, Kashyap S, Avati A, Harman S, Shaw H, Li R, Smith M, Shum K, Javitz J, Vetteth Y, Seto T, Bagley SC, Shah NH. A framework for making predictive models useful in practice. J Am Med Inform Assoc 2021; 28:1149-1158. [PMID: 33355350 PMCID: PMC8200271 DOI: 10.1093/jamia/ocaa318] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2020] [Accepted: 11/27/2020] [Indexed: 01/03/2023] Open
Abstract
OBJECTIVE To analyze the impact of factors in healthcare delivery on the net benefit of triggering an Advanced Care Planning (ACP) workflow based on predictions of 12-month mortality. MATERIALS AND METHODS We built a predictive model of 12-month mortality using electronic health record data and evaluated the impact of healthcare delivery factors on the net benefit of triggering an ACP workflow based on the models' predictions. Factors included nonclinical reasons that make ACP inappropriate: limited capacity for ACP, inability to follow up due to patient discharge, and availability of an outpatient workflow to follow up on missed cases. We also quantified the relative benefits of increasing capacity for inpatient ACP versus outpatient ACP. RESULTS Work capacity constraints and discharge timing can significantly reduce the net benefit of triggering the ACP workflow based on a model's predictions. However, the reduction can be mitigated by creating an outpatient ACP workflow. Given limited resources to either add capacity for inpatient ACP versus developing outpatient ACP capability, the latter is likely to provide more benefit to patient care. DISCUSSION The benefit of using a predictive model for identifying patients for interventions is highly dependent on the capacity to execute the workflow triggered by the model. We provide a framework for quantifying the impact of healthcare delivery factors and work capacity constraints on achieved benefit. CONCLUSION An analysis of the sensitivity of the net benefit realized by a predictive model triggered clinical workflow to various healthcare delivery factors is necessary for making predictive models useful in practice.
Collapse
Affiliation(s)
- Kenneth Jung
- Stanford Center for Biomedical Informatics, School of Medicine, Stanford University, Stanford, California, USA
| | - Sehj Kashyap
- Stanford Center for Biomedical Informatics, School of Medicine, Stanford University, Stanford, California, USA
| | - Anand Avati
- Department of Computer Science, School of Engineering, Stanford University, Stanford, California, USA
| | - Stephanie Harman
- Department of Medicine, School of Medicine, Stanford University, Stanford, California, USA
| | | | - Ron Li
- Department of Medicine, School of Medicine, Stanford University, Stanford, California, USA
| | - Margaret Smith
- Department of Medicine, School of Medicine, Stanford University, Stanford, California, USA
| | - Kenny Shum
- Department of Technology and Digital Solutions, Stanford Medicine, Stanford, California, USA
| | - Jacob Javitz
- Department of Technology and Digital Solutions, Stanford Medicine, Stanford, California, USA
| | - Yohan Vetteth
- Department of Technology and Digital Solutions, Stanford Medicine, Stanford, California, USA
| | - Tina Seto
- Department of Technology and Digital Solutions, Stanford Medicine, Stanford, California, USA
| | - Steven C Bagley
- Stanford Center for Biomedical Informatics, School of Medicine, Stanford University, Stanford, California, USA
| | - Nigam H Shah
- Stanford Center for Biomedical Informatics, School of Medicine, Stanford University, Stanford, California, USA
| |
Collapse
|
10
|
Fohner AE, Greene JD, Lawson BL, Chen JH, Kipnis P, Escobar GJ, Liu VX. Assessing clinical heterogeneity in sepsis through treatment patterns and machine learning. J Am Med Inform Assoc 2021; 26:1466-1477. [PMID: 31314892 DOI: 10.1093/jamia/ocz106] [Citation(s) in RCA: 40] [Impact Index Per Article: 13.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2019] [Revised: 05/24/2019] [Accepted: 05/29/2019] [Indexed: 12/22/2022] Open
Abstract
OBJECTIVE To use unsupervised topic modeling to evaluate heterogeneity in sepsis treatment patterns contained within granular data of electronic health records. MATERIALS AND METHODS A multicenter, retrospective cohort study of 29 253 hospitalized adult sepsis patients between 2010 and 2013 in Northern California. We applied an unsupervised machine learning method, Latent Dirichlet Allocation, to the orders, medications, and procedures recorded in the electronic health record within the first 24 hours of each patient's hospitalization to uncover empiric treatment topics across the cohort and to develop computable clinical signatures for each patient based on proportions of these topics. We evaluated how these topics correlated with common sepsis treatment and outcome metrics including inpatient mortality, time to first antibiotic, and fluids given within 24 hours. RESULTS Mean age was 70 ± 17 years with hospital mortality of 9.6%. We empirically identified 42 clinically recognizable treatment topics (eg, pneumonia, cellulitis, wound care, shock). Only 43.1% of hospitalizations had a single dominant topic, and a small minority (7.3%) had a single topic comprising at least 80% of their overall clinical signature. Across the entire sepsis cohort, clinical signatures were highly variable. DISCUSSION Heterogeneity in sepsis is a major barrier to improving targeted treatments, yet existing approaches to characterizing clinical heterogeneity are narrowly defined. A machine learning approach captured substantial patient- and population-level heterogeneity in treatment during early sepsis hospitalization. CONCLUSION Using topic modeling based on treatment patterns may enable more precise clinical characterization in sepsis and better understanding of variability in sepsis presentation and outcomes.
Collapse
Affiliation(s)
- Alison E Fohner
- Division of Research, Kaiser Permanente, Oakland, California, USA.,Department of Epidemiology, University of Washington, Seattle, Washington, USA
| | - John D Greene
- Division of Research, Kaiser Permanente, Oakland, California, USA
| | - Brian L Lawson
- Division of Research, Kaiser Permanente, Oakland, California, USA
| | - Jonathan H Chen
- Division of Biomedical Informatics Research, Stanford University, Stanford, California, USA
| | - Patricia Kipnis
- Division of Research, Kaiser Permanente, Oakland, California, USA
| | | | - Vincent X Liu
- Division of Research, Kaiser Permanente, Oakland, California, USA
| |
Collapse
|
11
|
Escobar GJ, Soltesz L, Schuler A, Niki H, Malenica I, Lee C. Prediction of obstetrical and fetal complications using automated electronic health record data. Am J Obstet Gynecol 2021; 224:137-147.e7. [PMID: 33098815 DOI: 10.1016/j.ajog.2020.10.030] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2020] [Revised: 10/06/2020] [Accepted: 10/20/2020] [Indexed: 12/23/2022]
Abstract
An increasing number of delivering women experience major morbidity and mortality. Limited work has been done on automated predictive models that could be used for prevention. Using only routinely collected obstetrical data, this study aimed to develop a predictive model suitable for real-time use with an electronic medical record. We used a retrospective cohort study design with split validation. The denominator consisted of women admitted to a delivery service. The numerator consisted of women who experienced a composite outcome that included both maternal (eg, uterine rupture, postpartum hemorrhage), fetal (eg, stillbirth), and neonatal (eg, hypoxic ischemic encephalopathy) adverse events. We employed machine learning methods, assessing model performance using the area under the receiver operator characteristic curve and number needed to evaluate. A total of 303,678 deliveries took place at 15 study hospitals between January 1, 2010, and March 31, 2018, and 4130 (1.36%) had ≥1 obstetrical complication. We employed data from 209,611 randomly selected deliveries (January 1, 2010, to March 31, 2017) as a derivation dataset and validated our findings on data from 52,398 randomly selected deliveries during the same time period (validation 1 dataset). We then applied our model to data from 41,669 deliveries from the last year of the study (April 1, 2017, to March 31, 2018 [validation 2 dataset]). Our model included 35 variables (eg, demographics, vital signs, laboratory tests, progress of labor indicators). In the validation 2 dataset, a gradient boosted model (area under the receiver operating characteristic curve or c statistic, 0.786) was slightly superior to a logistic regression model (c statistic, 0.778). Using an alert threshold of 4.1%, our final model would flag 16.7% of women and detect 52% of adverse outcomes, with a number needed to evaluate of 20.9 and 0.455 first alerts per day per 1000 annual deliveries. In conclusion, electronic medical record data can be used to predict obstetrical complications. The clinical utility of these automated models has not yet been demonstrated. To conduct interventions to assess whether using these models results in patient benefit, future work will need to focus on the development of clinical protocols suitable for use in interventions.
Collapse
Affiliation(s)
- Gabriel J Escobar
- Systems Research Initiative, Division of Research, Kaiser Permanente, Oakland, CA.
| | - Lauren Soltesz
- Systems Research Initiative, Division of Research, Kaiser Permanente, Oakland, CA
| | - Alejandro Schuler
- Systems Research Initiative, Division of Research, Kaiser Permanente, Oakland, CA
| | - Hamid Niki
- Perinatal Research Unit, Division of Research, Kaiser Permanente, Oakland, CA
| | - Ivana Malenica
- Division of Biostatistics, School of Public Health, University of California, Berkeley, Berkeley, CA
| | - Catherine Lee
- Systems Research Initiative, Division of Research, Kaiser Permanente, Oakland, CA
| |
Collapse
|
12
|
Lin DM, Peden CJ, Langness SM, Sammann A, Greenberg SB, Lane-Fall MB, Cooper JB. The Anesthesia Patient Safety Foundation Stoelting Conference 2019: Perioperative Deterioration-Early Recognition, Rapid Intervention, and the End of Failure-to-Rescue. Anesth Analg 2020; 131:e155-e159. [PMID: 33035027 DOI: 10.1213/ane.0000000000005008] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
Affiliation(s)
- Della M Lin
- Department of Surgery, John A. Burns School of Medicine, Honolulu, Hawaii,
| | - Carol J Peden
- Department of Anesthesiology, Keck School of Medicine at the University of Southern California, Los Angeles, California
| | | | - Amanda Sammann
- Department of Surgery, The Better Lab, San Francisco, California
| | - Steven B Greenberg
- Department of Anesthesiology, Critical Care and Pain Medicine, NorthShore University HealthSystem, Evanston, Illinois, Department of Anesthesiology and Critical Care, University of Chicago, Pritzker School of Medicine, Chicago, Illinois
| | - Meghan B Lane-Fall
- Department of Anesthesiology and Critical Care, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania
| | - Jeffrey B Cooper
- Department of Anesthesia, Critical Care & Pain Medicine, Massachusetts General Hospital, Boston, Massachusetts
| |
Collapse
|
13
|
Escobar GJ, Liu VX, Schuler A, Lawson B, Greene JD, Kipnis P. Automated Identification of Adults at Risk for In-Hospital Clinical Deterioration. N Engl J Med 2020; 383:1951-1960. [PMID: 33176085 PMCID: PMC7787261 DOI: 10.1056/nejmsa2001090] [Citation(s) in RCA: 144] [Impact Index Per Article: 36.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/31/2022]
Abstract
BACKGROUND Hospitalized adults whose condition deteriorates while they are in wards (outside the intensive care unit [ICU]) have considerable morbidity and mortality. Early identification of patients at risk for clinical deterioration has relied on manually calculated scores. Outcomes after an automated detection of impending clinical deterioration have not been widely reported. METHODS On the basis of a validated model that uses information from electronic medical records to identify hospitalized patients at high risk for clinical deterioration (which permits automated, real-time risk-score calculation), we developed an intervention program involving remote monitoring by nurses who reviewed records of patients who had been identified as being at high risk; results of this monitoring were then communicated to rapid-response teams at hospitals. We compared outcomes (including the primary outcome, mortality within 30 days after an alert) among hospitalized patients (excluding those in the ICU) whose condition reached the alert threshold at hospitals where the system was operational (intervention sites, where alerts led to a clinical response) with outcomes among patients at hospitals where the system had not yet been deployed (comparison sites, where a patient's condition would have triggered a clinical response after an alert had the system been operational). Multivariate analyses adjusted for demographic characteristics, severity of illness, and burden of coexisting conditions. RESULTS The program was deployed in a staggered fashion at 19 hospitals between August 1, 2016, and February 28, 2019. We identified 548,838 non-ICU hospitalizations involving 326,816 patients. A total of 43,949 hospitalizations (involving 35,669 patients) involved a patient whose condition reached the alert threshold; 15,487 hospitalizations were included in the intervention cohort, and 28,462 hospitalizations in the comparison cohort. Mortality within 30 days after an alert was lower in the intervention cohort than in the comparison cohort (adjusted relative risk, 0.84, 95% confidence interval, 0.78 to 0.90; P<0.001). CONCLUSIONS The use of an automated predictive model to identify high-risk patients for whom interventions by rapid-response teams could be implemented was associated with decreased mortality. (Funded by the Gordon and Betty Moore Foundation and others.).
Collapse
Affiliation(s)
- Gabriel J Escobar
- From the Systems Research Initiative, Kaiser Permanente Division of Research, Oakland (G.J.E., V.X.L., A.S., B.L., J.D.G., P.K.), the Intensive Care Unit, Kaiser Permanente Medical Center, Santa Clara (V.X.L.), and Unlearn.AI, San Francisco (A.S.) - all in California
| | - Vincent X Liu
- From the Systems Research Initiative, Kaiser Permanente Division of Research, Oakland (G.J.E., V.X.L., A.S., B.L., J.D.G., P.K.), the Intensive Care Unit, Kaiser Permanente Medical Center, Santa Clara (V.X.L.), and Unlearn.AI, San Francisco (A.S.) - all in California
| | - Alejandro Schuler
- From the Systems Research Initiative, Kaiser Permanente Division of Research, Oakland (G.J.E., V.X.L., A.S., B.L., J.D.G., P.K.), the Intensive Care Unit, Kaiser Permanente Medical Center, Santa Clara (V.X.L.), and Unlearn.AI, San Francisco (A.S.) - all in California
| | - Brian Lawson
- From the Systems Research Initiative, Kaiser Permanente Division of Research, Oakland (G.J.E., V.X.L., A.S., B.L., J.D.G., P.K.), the Intensive Care Unit, Kaiser Permanente Medical Center, Santa Clara (V.X.L.), and Unlearn.AI, San Francisco (A.S.) - all in California
| | - John D Greene
- From the Systems Research Initiative, Kaiser Permanente Division of Research, Oakland (G.J.E., V.X.L., A.S., B.L., J.D.G., P.K.), the Intensive Care Unit, Kaiser Permanente Medical Center, Santa Clara (V.X.L.), and Unlearn.AI, San Francisco (A.S.) - all in California
| | - Patricia Kipnis
- From the Systems Research Initiative, Kaiser Permanente Division of Research, Oakland (G.J.E., V.X.L., A.S., B.L., J.D.G., P.K.), the Intensive Care Unit, Kaiser Permanente Medical Center, Santa Clara (V.X.L.), and Unlearn.AI, San Francisco (A.S.) - all in California
| |
Collapse
|
14
|
|
15
|
What Do We Do After the Pilot Is Done? Implementation of a Hospital Early Warning System at Scale. Jt Comm J Qual Patient Saf 2020; 46:207-216. [PMID: 32085952 DOI: 10.1016/j.jcjq.2020.01.003] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/05/2019] [Revised: 01/15/2020] [Accepted: 01/16/2020] [Indexed: 11/22/2022]
Abstract
BACKGROUND Adults who deteriorate outside the ICU have high mortality. Most rapid response systems (RRSs) have employed manual detection processes that rapid response teams (RRTs) use to identify patients at risk. This project piloted the use of an automated early warning system (EWS), based on a very large database, that provides RRTs with 12 hours lead time to mount a response. Results from a 2-hospital pilot were encouraging, so leadership decided to deploy the Advance Alert Monitor (AAM) program in 19 more hospitals. CHALLENGE How can one deploy an RRS using an automated EWS at scale? SOLUTION EWS displays were removed from frontline clinicians' hospital electronic dashboards, and a Virtual Quality Team (VQT) RN was interposed between the EWS and the RRT. VQT RNs monitor the EWS remotely-when alerts are issued, they conduct a preliminary chart review and contact hospital RRT RNs. VQT and RRT RNs review the cases jointly. The RRT RNs then consult with hospitalists regarding clinical rescue and/or palliative care workflows. Subsequently, VQT RNs monitor patient charts, ensuring adherence to RRS practice standards. To enable this process, the project team developed a governance structure, clinical workflows, palliative care workflows, and documentation standards. RESULTS The AAM Program now functions in 21 Kaiser Permanente Northern California hospitals. VQT RNs monitor EWS alerts 24 hours a day, 7 days a week. The AAM Program handles ∼16,000 alerts per year. Its implementation has resulted in standardization of RRT staffing, clinical rescue workflows, and in-hospital palliative care.
Collapse
|
16
|
Liu VX, Bates DW, Wiens J, Shah NH. The number needed to benefit: estimating the value of predictive analytics in healthcare. J Am Med Inform Assoc 2019; 26:1655-1659. [PMID: 31192367 PMCID: PMC6857505 DOI: 10.1093/jamia/ocz088] [Citation(s) in RCA: 30] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/11/2019] [Revised: 05/10/2019] [Accepted: 05/17/2019] [Indexed: 12/21/2022] Open
Abstract
Predictive analytics in health care has generated increasing enthusiasm recently, as reflected in a rapidly growing body of predictive models reported in literature and in real-time embedded models using electronic health record data. However, estimating the benefit of applying any single model to a specific clinical problem remains challenging today. Developing a shared framework for estimating model value is therefore critical to facilitate the effective, safe, and sustainable use of predictive tools into the future. We highlight key concepts within the prediction-action dyad that together are expected to impact model benefit. These include factors relevant to model prediction (including the number needed to screen) as well as those relevant to the subsequent action (number needed to treat). In the simplest terms, a number needed to benefit contextualizes the numbers needed to screen and treat, offering an opportunity to estimate the value of a clinical predictive model in action.
Collapse
Affiliation(s)
- Vincent X Liu
- Division of Research, Kaiser Permanente, Oakland, California, USA
| | - David W Bates
- Division of General Internal Medicine, Brigham and Women’s Hospital, and Harvard Medical School, Boston, Massachusetts, USA
| | - Jenna Wiens
- Division of Computer Science and Engineering, College of Engineering, University of Michigan, Ann Arbor, Michigan, USA
| | - Nigam H Shah
- Division of Biomedical Informatics Research, Stanford University, Stanford, California, USA
| |
Collapse
|
17
|
Abstract
BACKGROUND Unplanned admissions of patients to intensive care units from medical-surgical units often result from failure to recognize clinical deterioration. The early warning score is a clinical decision support tool for nurse surveillance but must be communicated to nurses and implemented appropriately. A communication process including collaboration with experienced intensive care unit nurses may reduce unplanned transfers. OBJECTIVE To determine the impact of an early warning score communication bundle on medical-surgical transfers to the intensive care unit, rapid response team calls, and morbidity of patients upon intensive care unit transfer. METHODS After an early warning score was electronically embedded into medical records, a communication bundle including notification of and telephone collaboration between medical-surgical and intensive care unit nurses was implemented. Data were collected 3 months before and 21 months after implementation. RESULTS Rapid response team calls increased nonsignificantly during the study period (from 6.47 to 8.29 per 1000 patient-days). Rapid response team calls for patients with early warning scores greater than 4 declined (from 2.04 to 1.77 per 1000 patient-days). Intensive care unit admissions of patients after rapid response team calls significantly declined (P = .03), as did admissions of patients with early warning scores greater than 4 (P = .01), suggesting that earlier intervention for patient deterioration occurred. Documented reassessment response time declined significantly to 28 minutes (P = .002). CONCLUSION Electronic surveillance and collaboration with experienced intensive care unit nurses may improve care, control costs, and save lives. Critical care nurses have a role in coaching and guiding less experienced nurses.
Collapse
Affiliation(s)
- Cheryl Gagne
- Cheryl Gagne is vice president and Susan Fetzer is a nurse researcher in patient care services at Southern New Hampshire Medical Center, Nashua, New Hampshire
| | - Susan Fetzer
- Cheryl Gagne is vice president and Susan Fetzer is a nurse researcher in patient care services at Southern New Hampshire Medical Center, Nashua, New Hampshire
| |
Collapse
|
18
|
Escobar GJ, Gupta NR, Walsh EM, Soltesz L, Terry SM, Kipnis P. Automated early detection of obstetric complications: theoretic and methodologic considerations. Am J Obstet Gynecol 2019; 220:297-307. [PMID: 30682365 DOI: 10.1016/j.ajog.2019.01.208] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2018] [Revised: 12/20/2018] [Accepted: 01/10/2019] [Indexed: 12/01/2022]
Abstract
Compared with adults who are admitted to general medical-surgical wards, women who are admitted to labor and delivery services are at much lower risk of experiencing unexpected critical illness. Nonetheless, critical illness and other complications that put either the mother or fetus at risk do occur. One potential approach to prevention is to use automated early warning systems, such as those used for nonpregnant adults. Predictive models that use data extracted in real time from electronic records constitute the cornerstone of such systems. This article addresses several issues that are involved in the development of such predictive models: specification of temporal characteristics, choice of denominator, selection of outcomes for model calibration, potential uses of existing adult severity of illness scores, approaches to data processing, statistical considerations, validation, and options for instantiation. These have not been addressed explicitly in the obstetrics literature, which has focused on the use of manually assigned scores. In addition, this article provides some results from work in progress to develop 2 obstetric predictive models with the use of data from 262,071 women who were admitted to a labor and delivery service at 15 Kaiser Permanente Northern California hospitals between 2010 and 2017.
Collapse
Affiliation(s)
- Gabriel J Escobar
- Division of Research, Systems Research Initiative, Kaiser Permanente Northern California, Oakland, CA.
| | - Neeru R Gupta
- Department of Obstetrics and Gynecology, Kaiser Permanente Medical Center, Oakland, CA
| | - Eileen M Walsh
- Division of Research, Perinatal Research Unit, Kaiser Permanente Northern California, Oakland, CA
| | - Lauren Soltesz
- Division of Research, Systems Research Initiative, Kaiser Permanente Northern California, Oakland, CA
| | - Stephanie M Terry
- Department of Obstetrics and Gynecology, Kaiser Permanente Medical Center, San Francisco, CA
| | - Patricia Kipnis
- Division of Research, Systems Research Initiative, Kaiser Permanente Northern California, Oakland, CA; Decision Support, Kaiser Foundation Hospitals, Inc, Oakland, CA
| |
Collapse
|
19
|
[Consensus document for sepsis code implementation and development in the Community of Madrid]. REVISTA ESPANOLA DE QUIMIOTERAPIA : PUBLICACION OFICIAL DE LA SOCIEDAD ESPANOLA DE QUIMIOTERAPIA 2019; 32:400-409. [PMID: 31345006 PMCID: PMC6719654] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
Abstract
The consensus paper for the implementation and development of the sepsis code, finished in April 2017 is presented here. It was adopted by the Regional Office of Health as a working document for the implementation of the sepsis code in the Community of Madrid, both in the hospital setting (acute, middle and long-stay hospitals) and in Primary Care and Out-of-Hospital Emergency Services. It is now published without changes with respect to the original version, having only added the most significant bibliographical references. The document is divided into four parts: introduction, initial detection and assessment, early therapy and organizational recommendations. In the second to fourth sections, 25 statements or proposals have been included, agreed upon by the authors after several face-to-face meetings and an extensive "online" discussion. The annex includes nine tables that are intended as a practical guide to the activation of the sepsis code. Both the content of the recommendations and their formal writing have been made taking into account their applicability in all areas to which they are directed, which may have very different structural and functional characteristics and features, so that we have deliberately avoided a greater degree of concretion: the objective is not that the sepsis code is organized and applied identically in all of them, but that the health resources work in a coordinated manner aligned in the same direction.
Collapse
|
20
|
Op den Buijs J, Simons M, Golas S, Fischer N, Felsted J, Schertzer L, Agboola S, Kvedar J, Jethwani K. Predictive Modeling of 30-Day Emergency Hospital Transport of Patients Using a Personal Emergency Response System: Prognostic Retrospective Study. JMIR Med Inform 2018; 6:e49. [PMID: 30482741 PMCID: PMC6290270 DOI: 10.2196/medinform.9907] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/22/2018] [Revised: 07/20/2018] [Accepted: 08/07/2018] [Indexed: 11/13/2022] Open
Abstract
BACKGROUND Telehealth programs have been successful in reducing 30-day readmissions and emergency department visits. However, such programs often focus on the costliest patients with multiple morbidities and last for only 30 to 60 days postdischarge. Inexpensive monitoring of elderly patients via a personal emergency response system (PERS) to identify those at high risk for emergency hospital transport could be used to target interventions and prevent avoidable use of costly readmissions and emergency department visits after 30 to 60 days of telehealth use. OBJECTIVE The objectives of this study were to (1) develop and validate a predictive model of 30-day emergency hospital transport based on PERS data; and (2) compare the model's predictions with clinical outcomes derived from the electronic health record (EHR). METHODS We used deidentified medical alert pattern data from 290,434 subscribers to a PERS service to build a gradient tree boosting-based predictive model of 30-day hospital transport, which included predictors derived from subscriber demographics, self-reported medical conditions, caregiver network information, and up to 2 years of retrospective PERS medical alert data. We evaluated the model's performance on an independent validation cohort (n=289,426). We linked EHR and PERS records for 1815 patients from a home health care program to compare PERS-based risk scores with rates of emergency encounters as recorded in the EHR. RESULTS In the validation cohort, 2.22% (6411/289,426) of patients had 1 or more emergency transports in 30 days. The performance of the predictive model of emergency hospital transport, as evaluated by the area under the receiver operating characteristic curve, was 0.779 (95% CI 0.774-0.785). Among the top 1% of predicted high-risk patients, 25.5% had 1 or more emergency hospital transports in the next 30 days. Comparison with clinical outcomes from the EHR showed 3.9 times more emergency encounters among predicted high-risk patients than low-risk patients in the year following the prediction date. CONCLUSIONS Patient data collected remotely via PERS can be used to reliably predict 30-day emergency hospital transport. Clinical observations from the EHR showed that predicted high-risk patients had nearly four times higher rates of emergency encounters than did low-risk patients. Health care providers could benefit from our validated predictive model by targeting timely preventive interventions to high-risk patients. This could lead to overall improved patient experience, higher quality of care, and more efficient resource utilization.
Collapse
Affiliation(s)
| | | | - Sara Golas
- Partners HealthCare Pivot Labs, Partners HealthCare, Boston, MA, United States
| | - Nils Fischer
- Partners HealthCare Pivot Labs, Partners HealthCare, Boston, MA, United States
| | - Jennifer Felsted
- Partners HealthCare Pivot Labs, Partners HealthCare, Boston, MA, United States.,Department of Dermatology, Harvard Medical School, Boston, MA, United States
| | | | - Stephen Agboola
- Partners HealthCare Pivot Labs, Partners HealthCare, Boston, MA, United States.,Department of Dermatology, Harvard Medical School, Boston, MA, United States.,Department of Dermatology, Massachusetts General Hospital, Boston, MA, United States
| | - Joseph Kvedar
- Department of Dermatology, Harvard Medical School, Boston, MA, United States.,Department of Dermatology, Massachusetts General Hospital, Boston, MA, United States.,Partners Connected Health, Partners HealthCare, Boston, MA, United States
| | - Kamal Jethwani
- Partners HealthCare Pivot Labs, Partners HealthCare, Boston, MA, United States.,Department of Dermatology, Harvard Medical School, Boston, MA, United States.,Department of Dermatology, Massachusetts General Hospital, Boston, MA, United States
| |
Collapse
|
21
|
The impact of delayed critical care outreach team activation on in-hospital mortality and other patient outcomes: a historical cohort study. Can J Anaesth 2018; 65:1210-1217. [PMID: 29980998 DOI: 10.1007/s12630-018-1180-5] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2018] [Revised: 04/15/2018] [Accepted: 04/25/2018] [Indexed: 10/28/2022] Open
Abstract
PURPOSE Early warning scores (EWS) and critical care outreach teams (CCOT) have been developed to respond to decompensating patients. Nevertheless, controversy exists around their effectiveness. The primary objective of this study was to determine if a delay of ≥ 60 min between when a patient was identified as meeting EWS criteria and the CCOT was activated impacted in-hospital mortality. METHODS This was a historical cohort study evaluating all new CCOT activations over a four-year study period (1 June 2007 to 31 August 2011) for inpatients ≥ 18 yr of age at two academic tertiary care hospitals in London, Ontario, Canada. Multivariable logistic regression accounting for repeated measures was used to determine the effect of delayed CCOT activation on in-hospital mortality (primary outcome). Differences in outcomes between medical and surgical patients were also examined. RESULTS There were 3,133 CCOT activations for 1,684 (53.8%) medical patients and 1,449 (46.2%) surgical patients during the study period. The CCOT was activated < 60 min of a patient meeting EWS criteria in 2,160 (68.9%) cases and ≥ 60 min in 973 (31.1%) cases. Patients with ≥ 60 min delay were more likely be admitted to the intensive care unit (odds ratio [OR], 1.22; 95% confidence interval [CI], 1.07 to 1.47) and to suffer in-hospital mortality (OR, 1.30; 95% CI, 1.08 to 1.56). Irrespective of delay, surgical patients were less likely to experience in-hospital mortality than medical patients (OR, 0.46; 95% CI, 0.39 to 0.55). CONCLUSION Including the rates of delay in CCOT activation and the admitting service could be an additional step in exploring the conflicting results seen in the current literature assessing the impact of CCOT on patient outcomes.
Collapse
|
22
|
Granich R, Sutton Z, Kim YS, Anderson M, Wood H, Scharf JE, Ragins A, Escobar GJ. Early detection of critical illness outside the intensive care unit: Clarifying treatment plans and honoring goals of care using a supportive care team. J Hosp Med 2016; 11 Suppl 1:S40-S47. [PMID: 27805799 DOI: 10.1002/jhm.2660] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/11/2016] [Revised: 09/06/2016] [Accepted: 09/11/2016] [Indexed: 11/10/2022]
Abstract
Given the high mortality experienced by patients who deteriorate outside the intensive care unit, issues related to patient preferences around escalation of care are common. However, the literature on early warning systems (EWSs) provides limited information on how respecting patient preferences can be incorporated into clinical workflows. In this report, we describe how we developed workflows for integrating supportive care with an automated EWS in the context of a 2-hospital pilot. We used the Institute for Healthcare Improvement's Plan-Do-Study-Act approach to achieve consensus with clinicians and administrators. The workflows will serve as the basis for dissemination to an additional 19 hospitals. We were successful in integrating an automated EWS with supportive care. Our workflows take local resource availability into account and have been well received by hospitalists, nurses, and families. Our work demonstrates that one can achieve integration of proactive supportive care into the operation of an EWS. Creation of a palliative care response arm that is complementary to a clinical rescue arm ensures that patient preferences are respected. Journal of Hospital Medicine 2016;11:S40-S47. © 2016 Society of Hospital Medicine.
Collapse
Affiliation(s)
- Russ Granich
- Kaiser Permanente South San Francisco Medical Center, South San Francisco, California.
| | - Zoe Sutton
- Kaiser Permanente South San Francisco Medical Center, South San Francisco, California
| | - Yan S Kim
- Systems Research Initiative, Division of Research, Kaiser Permanente Northern California, Oakland, California
| | - Megan Anderson
- Kaiser Permanente Sacramento Medical Center, Sacramento, California
| | - Helen Wood
- Kaiser Permanente South San Francisco Medical Center, South San Francisco, California
| | - John E Scharf
- Kaiser Permanente Sacramento Medical Center, Sacramento, California
| | - Arona Ragins
- Systems Research Initiative, Division of Research, Kaiser Permanente Northern California, Oakland, California
| | - Gabriel J Escobar
- Systems Research Initiative, Division of Research, Kaiser Permanente Northern California, Oakland, California
| |
Collapse
|
23
|
Escobar GJ, Turk BJ, Ragins A, Ha J, Hoberman B, LeVine SM, Ballesca MA, Liu V, Kipnis P. Piloting electronic medical record-based early detection of inpatient deterioration in community hospitals. J Hosp Med 2016; 11 Suppl 1:S18-S24. [PMID: 27805795 PMCID: PMC5510649 DOI: 10.1002/jhm.2652] [Citation(s) in RCA: 60] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/11/2016] [Revised: 08/15/2016] [Accepted: 08/24/2016] [Indexed: 11/12/2022]
Abstract
Patients who deteriorate in the hospital outside the intensive care unit (ICU) have higher mortality and morbidity than those admitted directly to the ICU. As more hospitals deploy comprehensive inpatient electronic medical records (EMRs), attempts to support rapid response teams with automated early detection systems are becoming more frequent. We aimed to describe some of the technical and operational challenges involved in the deployment of an early detection system. This 2-hospital pilot, set within an integrated healthcare delivery system with 21 hospitals, had 2 objectives. First, it aimed to demonstrate that severity scores and probability estimates could be provided to hospitalists in real time. Second, it aimed to surface issues that would need to be addressed so that deployment of the early warning system could occur in all remaining hospitals. To achieve these objectives, we first established a rationale for the development of an early detection system through the analysis of risk-adjusted outcomes. We then demonstrated that EMR data could be employed to predict deteriorations. After addressing specific organizational mandates (eg, defining the clinical response to a probability estimate), we instantiated a set of equations into a Java application that transmits scores and probability estimates so that they are visible in a commercially available EMR every 6 hours. The pilot has been successful and deployment to the remaining hospitals has begun. Journal of Hospital Medicine 2016;11:S18-S24. © 2016 Society of Hospital Medicine.
Collapse
Affiliation(s)
- Gabriel J Escobar
- Division of Research, Kaiser Permanente Northern California, Oakland, California. , , , ,
- Department of Inpatient Pediatrics, Kaiser Permanente Medical Center, Walnut Creek, California. , , , ,
| | - Benjamin J Turk
- Division of Research, Kaiser Permanente Northern California, Oakland, California
| | - Arona Ragins
- Division of Research, Kaiser Permanente Northern California, Oakland, California
| | - Jason Ha
- IMG-Systems Integration, Kaiser Permanente, Pleasanton, California
| | | | | | | | - Vincent Liu
- Division of Research, Kaiser Permanente Northern California, Oakland, California
- Department of Critical Care, Kaiser Permanente Medical Center, Santa Clara, California
| | - Patricia Kipnis
- Decision Support, Kaiser Foundation Health Plan, Oakland, California
| |
Collapse
|