1
|
Wan YKJ, Wright MC, McFarland MM, Dishman D, Nies MA, Rush A, Madaras-Kelly K, Jeppesen A, Del Fiol G. Information displays for automated surveillance algorithms of in-hospital patient deterioration: a scoping review. J Am Med Inform Assoc 2023; 31:256-273. [PMID: 37847664 PMCID: PMC10746326 DOI: 10.1093/jamia/ocad203] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/20/2023] [Revised: 09/12/2023] [Accepted: 10/02/2023] [Indexed: 10/19/2023] Open
Abstract
OBJECTIVE Surveillance algorithms that predict patient decompensation are increasingly integrated with clinical workflows to help identify patients at risk of in-hospital deterioration. This scoping review aimed to identify the design features of the information displays, the types of algorithm that drive the display, and the effect of these displays on process and patient outcomes. MATERIALS AND METHODS The scoping review followed Arksey and O'Malley's framework. Five databases were searched with dates between January 1, 2009 and January 26, 2022. Inclusion criteria were: participants-clinicians in inpatient settings; concepts-intervention as deterioration information displays that leveraged automated AI algorithms; comparison as usual care or alternative displays; outcomes as clinical, workflow process, and usability outcomes; and context as simulated or real-world in-hospital settings in any country. Screening, full-text review, and data extraction were reviewed independently by 2 researchers in each step. Display categories were identified inductively through consensus. RESULTS Of 14 575 articles, 64 were included in the review, describing 61 unique displays. Forty-one displays were designed for specific deteriorations (eg, sepsis), 24 provided simple alerts (ie, text-based prompts without relevant patient data), 48 leveraged well-accepted score-based algorithms, and 47 included nurses as the target users. Only 1 out of the 10 randomized controlled trials reported a significant effect on the primary outcome. CONCLUSIONS Despite significant advancements in surveillance algorithms, most information displays continue to leverage well-understood, well-accepted score-based algorithms. Users' trust, algorithmic transparency, and workflow integration are significant hurdles to adopting new algorithms into effective decision support tools.
Collapse
Affiliation(s)
- Yik-Ki Jacob Wan
- Department of Biomedical Informatics, University of Utah, Salt Lake City, UT 84108, United States
| | - Melanie C Wright
- College of Pharmacy, Idaho State University, Meridian, ID 83642, United States
| | - Mary M McFarland
- Eccles Health Sciences Library, University of Utah, Salt Lake City, UT 84112, United States
| | - Deniz Dishman
- Cizik School of Nursing Department of Research, University of Texas Health Science Center at Houston, Houston, TX 77030, United States
| | - Mary A Nies
- College of Health, Idaho State University, Pocatello, ID 83209, United States
| | - Adriana Rush
- Department of Biomedical Informatics, University of Utah, Salt Lake City, UT 84108, United States
| | - Karl Madaras-Kelly
- College of Pharmacy, Idaho State University, Meridian, ID 83642, United States
| | - Amanda Jeppesen
- College of Pharmacy, Idaho State University, Meridian, ID 83642, United States
| | - Guilherme Del Fiol
- Department of Biomedical Informatics, University of Utah, Salt Lake City, UT 84108, United States
| |
Collapse
|
2
|
Ackermann K, Baker J, Green M, Fullick M, Varinli H, Westbrook J, Li L. Computerized Clinical Decision Support Systems for the Early Detection of Sepsis Among Adult Inpatients: Scoping Review. J Med Internet Res 2022; 24:e31083. [PMID: 35195528 PMCID: PMC8908200 DOI: 10.2196/31083] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2021] [Revised: 09/23/2021] [Accepted: 10/29/2021] [Indexed: 12/21/2022] Open
Abstract
Background Sepsis is a significant cause of morbidity and mortality worldwide. Early detection of sepsis followed promptly by treatment initiation improves patient outcomes and saves lives. Hospitals are increasingly using computerized clinical decision support (CCDS) systems for the rapid identification of adult patients with sepsis. Objective This scoping review aims to systematically describe studies reporting on the use and evaluation of CCDS systems for the early detection of adult inpatients with sepsis. Methods The protocol for this scoping review was previously published. A total of 10 electronic databases (MEDLINE, Embase, CINAHL, the Cochrane database, LILACS [Latin American and Caribbean Health Sciences Literature], Scopus, Web of Science, OpenGrey, ClinicalTrials.gov, and PQDT [ProQuest Dissertations and Theses]) were comprehensively searched using terms for sepsis, CCDS, and detection to identify relevant studies. Title, abstract, and full-text screening were performed by 2 independent reviewers using predefined eligibility criteria. Data charting was performed by 1 reviewer with a second reviewer checking a random sample of studies. Any disagreements were discussed with input from a third reviewer. In this review, we present the results for adult inpatients, including studies that do not specify patient age. Results A search of the electronic databases retrieved 12,139 studies following duplicate removal. We identified 124 studies for inclusion after title, abstract, full-text screening, and hand searching were complete. Nearly all studies (121/124, 97.6%) were published after 2009. Half of the studies were journal articles (65/124, 52.4%), and the remainder were conference abstracts (54/124, 43.5%) and theses (5/124, 4%). Most studies used a single cohort (54/124, 43.5%) or before-after (42/124, 33.9%) approach. Across all 124 included studies, patient outcomes were the most frequently reported outcomes (107/124, 86.3%), followed by sepsis treatment and management (75/124, 60.5%), CCDS usability (14/124, 11.3%), and cost outcomes (9/124, 7.3%). For sepsis identification, the systemic inflammatory response syndrome criteria were the most commonly used, alone (50/124, 40.3%), combined with organ dysfunction (28/124, 22.6%), or combined with other criteria (23/124, 18.5%). Over half of the CCDS systems (68/124, 54.8%) were implemented alongside other sepsis-related interventions. Conclusions The current body of literature investigating the implementation of CCDS systems for the early detection of adult inpatients with sepsis is extremely diverse. There is substantial variability in study design, CCDS criteria and characteristics, and outcomes measured across the identified literature. Future research on CCDS system usability, cost, and impact on sepsis morbidity is needed. International Registered Report Identifier (IRRID) RR2-10.2196/24899
Collapse
Affiliation(s)
- Khalia Ackermann
- Centre for Health Systems and Safety Research, Australian Institute of Health Innovation, Macquarie University, Australia
| | - Jannah Baker
- Centre for Health Systems and Safety Research, Australian Institute of Health Innovation, Macquarie University, Australia
| | | | - Mary Fullick
- Clinical Excellence Commission, Sydney, Australia
| | | | - Johanna Westbrook
- Centre for Health Systems and Safety Research, Australian Institute of Health Innovation, Macquarie University, Australia
| | - Ling Li
- Centre for Health Systems and Safety Research, Australian Institute of Health Innovation, Macquarie University, Australia
| |
Collapse
|
3
|
Wronikowska MW, Malycha J, Morgan LJ, Westgate V, Petrinic T, Young JD, Watkinson PJ. Systematic review of applied usability metrics within usability evaluation methods for hospital electronic healthcare record systems: Metrics and Evaluation Methods for eHealth Systems. J Eval Clin Pract 2021; 27:1403-1416. [PMID: 33982356 PMCID: PMC9438452 DOI: 10.1111/jep.13582] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/18/2020] [Revised: 04/19/2021] [Accepted: 04/20/2021] [Indexed: 01/01/2023]
Abstract
BACKGROUND AND OBJECTIVES Electronic healthcare records have become central to patient care. Evaluation of new systems include a variety of usability evaluation methods or usability metrics (often referred to interchangeably as usability components or usability attributes). This study reviews the breadth of usability evaluation methods, metrics, and associated measurement techniques that have been reported to assess systems designed for hospital staff to assess inpatient clinical condition. METHODS Following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology, we searched Medline, EMBASE, CINAHL, Cochrane Database of Systematic Reviews, and Open Grey from 1986 to 2019. For included studies, we recorded usability evaluation methods or usability metrics as appropriate, and any measurement techniques applied to illustrate these. We classified and described all usability evaluation methods, usability metrics, and measurement techniques. Study quality was evaluated using a modified Downs and Black checklist. RESULTS The search identified 1336 studies. After abstract screening, 130 full texts were reviewed. In the 51 included studies 11 distinct usability evaluation methods were identified. Within these usability evaluation methods, seven usability metrics were reported. The most common metrics were ISO9241-11 and Nielsen's components. An additional "usefulness" metric was reported in almost 40% of included studies. We identified 70 measurement techniques used to evaluate systems. Overall study quality was reflected in a mean modified Downs and Black checklist score of 6.8/10 (range 1-9) 33% studies classified as "high-quality" (scoring eight or higher), 51% studies "moderate-quality" (scoring 6-7), and the remaining 16% (scoring below five) were "low-quality." CONCLUSION There is little consistency within the field of electronic health record systems evaluation. This review highlights the variability within usability methods, metrics, and reporting. Standardized processes may improve evaluation and comparison electronic health record systems and improve their development and implementation.
Collapse
Affiliation(s)
| | - James Malycha
- Critical Care Research Group, Nuffield Department of Clinical NeurosciencesUniversity of OxfordOxfordUK
- Department of Acute Care MedicineUniversity of AdelaideAdelaideAustralia
| | - Lauren J. Morgan
- Nuffield Department of Surgical SciencesUniversity of Oxford, John Radcliffe HospitalOxfordUK
| | - Verity Westgate
- Critical Care Research Group, Nuffield Department of Clinical NeurosciencesUniversity of OxfordOxfordUK
| | - Tatjana Petrinic
- Bodleian Health Care LibrariesJohn Radcliffe Hospital, University of OxfordOxfordUK
| | - J Duncan Young
- Critical Care Research Group, Nuffield Department of Clinical NeurosciencesUniversity of OxfordOxfordUK
| | - Peter J. Watkinson
- Critical Care Research Group, Nuffield Department of Clinical NeurosciencesUniversity of OxfordOxfordUK
| |
Collapse
|
4
|
Duffy A, Christie G, Moreno S. Examining Challenges to the Incorporation of End Users in the Design of Digital Health Interventions: Protocol for a Systematic Review. JMIR Res Protoc 2021; 10:e28083. [PMID: 34309578 PMCID: PMC8367163 DOI: 10.2196/28083] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/19/2021] [Revised: 06/01/2021] [Accepted: 06/21/2021] [Indexed: 11/20/2022] Open
Abstract
Background The process of designing a digital health intervention (DHI)—also referred to as mobile health or eHealth—spans needs assessments, technical functionality and feasibility, user satisfaction, effectiveness, impact, and value. These interventions are causing a rapid evolution in the landscape of health care. Multiple studies have shown their propensity to extend both the quality and reach of interventions. However, failure to improve DHI design is linked to failed uptake and health outcomes. This dilemma is further conflicted by the colliding backdrops of the digital and health industries, both of which approach, understand, and involve end users differently in the framing of a DHI. Objective The objective of this systematic review is to assess the challenges to incorporating end users in the design stage of digital health interventions, to identify key pain points, and to identify limitations and gaps for areas of future investigation. Methods The PRISMA-P (Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols) checklist will be used to structure this protocol. A systematic search of the PsycINFO, PubMed (MEDLINE), Web of Science, CINAHL, Scopus, and IEEE Xplore databases will be conducted. Additionally, the PerSPEcTiF guidelines for complex interventions will be consulted. Two reviewers will independently screen the titles and abstracts of the identified references and select studies according to the eligibility criteria. Any discrepancies will then be discussed and resolved. Two reviewers will independently extract and validate data from the included studies into a standardized form and conduct quality appraisal. Results As of February 2021, we have completed a preliminary literature search examining challenges to the incorporation of end users in the design stage of DHIs. Systematic searches, data extraction and analysis, and writing of the systematic review are expected to be completed by December 2021. Conclusions This systematic review aims to provide an effective summary of key pain points toward incorporating end users in DHIs. Results from this review will provide an evidence base for a better approach to end user involvement in the interest of improving efficacy and uptake of DHIs. Trial Registration PROSPERO International Prospective Register of Systematic Reviews CRD42021238164; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=238164 International Registered Report Identifier (IRRID) PRR1-10.2196/28083
Collapse
Affiliation(s)
- Anthony Duffy
- School of Interactive Arts & Technology, Simon Fraser University, Surrey, BC, Canada
| | | | - Sylvain Moreno
- School of Interactive Arts & Technology, Simon Fraser University, Surrey, BC, Canada
| |
Collapse
|
5
|
Sakib N, Ahamed SI, Khan RA, Griffin PM, Haque MM. Unpacking Prevalence and Dichotomy in Quick Sequential Organ Failure Assessment and Systemic Inflammatory Response Syndrome Parameters: Observational Data-Driven Approach Backed by Sepsis Pathophysiology. JMIR Med Inform 2020; 8:e18352. [PMID: 33270030 PMCID: PMC7746497 DOI: 10.2196/18352] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2020] [Revised: 08/10/2020] [Accepted: 09/15/2020] [Indexed: 12/29/2022] Open
Abstract
BACKGROUND Considering morbidity, mortality, and annual treatment costs, the dramatic rise in the incidence of sepsis and septic shock among intensive care unit (ICU) admissions in US hospitals is an increasing concern. Recent changes in the sepsis definition (sepsis-3), based on the quick Sequential Organ Failure Assessment (qSOFA), have motivated the international medical informatics research community to investigate score recalculation and information retrieval, and to study the intersection between sepsis-3 and the previous definition (sepsis-2) based on systemic inflammatory response syndrome (SIRS) parameters. OBJECTIVE The objective of this study was three-fold. First, we aimed to unpack the most prevalent criterion for sepsis (for both sepsis-3 and sepsis-2 predictors). Second, we intended to determine the most prevalent sepsis scenario in the ICU among 4 possible scenarios for qSOFA and 11 possible scenarios for SIRS. Third, we investigated the multicollinearity or dichotomy among qSOFA and SIRS predictors. METHODS This observational study was conducted according to the most recent update of Medical Information Mart for Intensive Care (MIMIC-III, Version 1.4), the critical care database developed by MIT. The qSOFA (sepsis-3) and SIRS (sepsis-2) parameters were analyzed for patients admitted to critical care units from 2001 to 2012 in Beth Israel Deaconess Medical Center (Boston, MA, USA) to determine the prevalence and underlying relation between these parameters among patients undergoing sepsis screening. We adopted a multiblind Delphi method to seek a rationale for decisions in several stages of the research design regarding handling missing data and outlier values, statistical imputations and biases, and generalizability of the study. RESULTS Altered mental status in the Glasgow Coma Scale (59.28%, 38,854/65,545 observations) was the most prevalent sepsis-3 (qSOFA) criterion and the white blood cell count (53.12%, 17,163/32,311 observations) was the most prevalent sepsis-2 (SIRS) criterion confronted in the ICU. In addition, the two-factored sepsis criterion of high respiratory rate (≥22 breaths/minute) and altered mental status (28.19%, among four possible qSOFA scenarios besides no sepsis) was the most prevalent sepsis-3 (qSOFA) scenario, and the three-factored sepsis criterion of tachypnea, high heart rate, and high white blood cell count (12.32%, among 11 possible scenarios besides no sepsis) was the most prevalent sepsis-2 (SIRS) scenario in the ICU. Moreover, the absolute Pearson correlation coefficients were not significant, thereby nullifying the likelihood of any linear correlation among the critical parameters and assuring the lack of multicollinearity between the parameters. Although this further bolsters evidence for their dichotomy, the absence of multicollinearity cannot guarantee that two random variables are statistically independent. CONCLUSIONS Quantifying the prevalence of the qSOFA criteria of sepsis-3 in comparison with the SIRS criteria of sepsis-2, and understanding the underlying dichotomy among these parameters provides significant inferences for sepsis treatment initiatives in the ICU and informing hospital resource allocation. These data-driven results further offer design implications for multiparameter intelligent sepsis prediction in the ICU.
Collapse
Affiliation(s)
- Nazmus Sakib
- Ubicomp Lab, Department of Computer Science, Marquette University, Milwaukee, WI, United States
| | - Sheikh Iqbal Ahamed
- Ubicomp Lab, Department of Computer Science, Marquette University, Milwaukee, WI, United States
| | - Rumi Ahmed Khan
- College of Medicine, University of Central Florida, Orlando, FL, United States
| | - Paul M Griffin
- Regenstrief Center for Healthcare Engineering, Purdue University, West Lafayette, IN, United States
| | - Md Munirul Haque
- RB Annis School of Engineering, University of Indianapolis, Indianapolis, IN, United States
| |
Collapse
|
6
|
Li L, Ackermann K, Baker J, Westbrook J. Use and Evaluation of Computerized Clinical Decision Support Systems for Early Detection of Sepsis in Hospitals: Protocol for a Scoping Review. JMIR Res Protoc 2020; 9:e24899. [PMID: 33215998 PMCID: PMC7718090 DOI: 10.2196/24899] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2020] [Revised: 11/02/2020] [Accepted: 11/03/2020] [Indexed: 12/29/2022] Open
Abstract
BACKGROUND Sepsis is a leading cause of death in hospitals, with high associated costs for both patients and health care systems worldwide. Early detection followed by timely intervention is critical for successful sepsis management and, hence, can save lives. Health care institutions are increasingly leveraging clinical data captured in electronic health records for the development of computerized clinical decision support (CCDS) systems aimed at enhancing the early detection of sepsis. However, a comprehensive evidence base regarding sepsis CCDS systems to inform clinical practice, research, and policy is currently lacking. OBJECTIVE This scoping review aims to systematically describe studies reporting on the use and evaluation of CCDS systems for early detection of sepsis in hospitals. METHODS The methodology for conducting scoping reviews presented by the Joanna Briggs Institute Reviewer's Manual and the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) will be used and adapted as guides. A comprehensive literature search of 10 electronic databases will be conducted to identify all empirical quantitative and qualitative studies that investigate the use of CCDS systems for early detection of sepsis in hospitals. Detailed inclusion and exclusion criteria have been developed. Two reviewers will independently screen all articles based on these criteria. Any discrepancies will be resolved through discussion and further review by a third researcher if required. RESULTS Electronic database searches have retrieved 12,139 references after removing 10,051 duplicates. As of the submission date of this protocol, we have completed the title and abstract screening. A total of 372 references will be included for full-text screening. Only 15.9% (59/372) of these studies were focused on children: 11.0% (41/372) for pediatric and 4.8% (18/372) for neonatal patients. The scoping review and the manuscript will be completed by December 2020. CONCLUSIONS Results of this review will guide researchers in determining gaps and shortcomings in the current evidence base for CCDS system use and evaluation in the early detection of sepsis. The findings will be shared with key stakeholders in clinical care, research, policy, and patient advocacy. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID) PRR1-10.2196/24899.
Collapse
Affiliation(s)
- Ling Li
- Centre for Health Systems and Safety Research, Australian Institute of Health Innovation, Macquarie University, North Ryde, Australia
| | - Khalia Ackermann
- Centre for Health Systems and Safety Research, Australian Institute of Health Innovation, Macquarie University, North Ryde, Australia
| | - Jannah Baker
- Centre for Health Systems and Safety Research, Australian Institute of Health Innovation, Macquarie University, North Ryde, Australia
| | - Johanna Westbrook
- Centre for Health Systems and Safety Research, Australian Institute of Health Innovation, Macquarie University, North Ryde, Australia
| |
Collapse
|
7
|
Abstract
Sepsis is a dysregulated immune response to an infection that leads to organ dysfunction. Knowledge of the pathophysiology of organ failure in sepsis is crucial for optimizing the management and treatment of patients and for the development of potential new therapies. In clinical practice, six major organ systems - the cardiovascular (including the microcirculation), respiratory, renal, neurological, haematological and hepatic systems - can be assessed and monitored, whereas others, such as the gut, are less accessible. Over the past 2 decades, considerable amounts of new data have helped improve our understanding of sepsis pathophysiology, including the regulation of inflammatory pathways and the role played by immune suppression during sepsis. The effects of impaired cellular function, including mitochondrial dysfunction and altered cell death mechanisms, on the development of organ dysfunction are also being unravelled. Insights have been gained into interactions between key organs (such as the kidneys and the gut) and organ-organ crosstalk during sepsis. The important role of the microcirculation in sepsis is increasingly apparent, and new techniques have been developed that make it possible to visualize the microcirculation at the bedside, although these techniques are only research tools at present.
Collapse
Affiliation(s)
- Christophe Lelubre
- Laboratoire de Médecine Expérimentale (ULB 222 Unit), Université Libre de Bruxelles, CHU de Charleroi, A. Vésale Hospital, Montigny-Le-Tilleul, Belgium.,Department of Internal Medicine, CHU Charleroi - Hôpital Civil Marie Curie, Lodelinsart, Belgium
| | - Jean-Louis Vincent
- Department of Intensive Care, Erasme Hospital, Université libre de Bruxelles, Brussels, Belgium.
| |
Collapse
|