Use and Evaluation of Computerized Clinical Decision Support Systems for Early Detection of Sepsis in Hospitals: Protocol for a Scoping Review

A scoping review on pediatric sepsis prediction technologies in healthcare

This scoping review evaluates recent advancements in data-driven technologies for predicting non-neonatal pediatric sepsis, including artificial intelligence, machine learning, and other methodologies. Of the 27 included studies, 23 (85%) were single-center investigations, and 16 (59%) used logistic regression. Notably, 20 (74%) studies used datasets with a low prevalence of sepsis-related outcomes, with area under the receiver operating characteristic scores ranging from 0.56 to 0.99. Prediction time points varied widely, and development characteristics, performance metrics, implementation outcomes, and considerations for human factors-especially workflow integration and clinical judgment-were inconsistently reported. The variations in endpoint definitions highlight the potential significance of the 2024 consensus criteria in future development. Future research should strengthen the involvement of clinical users to enhance the understanding and integration of human factors in designing and evaluating these technologies, ultimately aiming for safe and effective integration in pediatric healthcare.

Evaluation of a Mobile-Based Immunization Decision Support System for Scheduling Age-Appropriate Vaccine Schedules for Children Younger Than 2 Years in Pakistan and Bangladesh: Lessons From a Multisite, Mixed Methods Study

BACKGROUND

Missed opportunities for vaccination (MOVs), that is, when children interact with the health system but fail to receive age-eligible vaccines, pose a crucial challenge for equitable and universal immunization coverage. Inaccurate interpretations of complex catch-up schedules by health workers contribute to MOVs.

OBJECTIVE

We assessed the feasibility of a mobile-based immunization decision support system (iDSS) to automatically construct age-appropriate vaccination schedules for children and to prevent MOVs.

METHODS

A sequential exploratory mixed methods study was conducted at 6 immunization centers in Pakistan and Bangladesh. An android-based iDSS that is packaged in the form of an application programming interface constructed age-appropriate immunization schedules for eligible children. The diagnostic accuracy of the iDSS was measured by comparing the schedules constructed by the iDSS with the gold standard of evaluation (World Health Organization-recommended Expanded Programme on Immunization schedule constructed by a vaccines expert). Preliminary estimates were collected on the number of MOVs among visiting children (caused by inaccurate vaccination scheduling by vaccinators) that could be reduced through iDSS by comparing the manual schedules constructed by vaccinators with the gold standard. Finally, the vaccinators' understanding, perceived usability, and acceptability of the iDSS were determined through interviews with key informants.

RESULTS

From July 5, 2019, to April 11, 2020, a total of 6241 immunization visits were recorded from 4613 eligible children. Data were collected for 17,961 immunization doses for all antigens. The iDSS correctly scheduled 99.8% (17,932/17,961) of all age-appropriate immunization doses compared with the gold standard. In comparison, vaccinators correctly scheduled 96.8% (17,378/17,961) of all immunization doses. A total of 3.2% (583/17,961) of all due doses (across antigens) were missed in age-eligible children by the vaccinators across both countries. Vaccinators reported positively on the usefulness of iDSS, as well as the understanding and benefits of the technology.

CONCLUSIONS

This study demonstrated the feasibility of a mobile-based iDSS to accurately construct age-appropriate vaccination schedules for children aged 0 to 23 months across multicountry and low- and middle-income country settings, and underscores its potential to increase immunization coverage and timeliness by eliminating MOVs.

Early Identification of Severe Sepsis in Pediatric Patients Using an Electronic Alert System

INTRODUCTION

Prompt sepsis recognition and the initiation of standardized treatment bundles lead to improved outcomes. We developed automated severe sepsis alerts through the electronic medical record and paging system to aid clinicians in rapidly identifying pediatric patients with severe sepsis in our emergency department and inpatient units. Our Specific, Measurable, Applicable, Realistic, Timely aim was to improve 1-hour severe sepsis treatment bundle compliance to 60% with these electronic interruptive alerts.

METHODS

We developed the alert's criteria based on the 2005 International Pediatric Sepsis Consensus definitions. We performed 2 interventions: requiring the bedside nurse to answer the already implemented nurse-targeted (NT) severe sepsis alert, and the implementation of the physician-targeted (PT) severe sepsis alert. When systemic inflammatory response syndrome criteria were met, the NT alert triggered, and when organ dysfunction was also identified, an interruptive PT alert triggered, and the respective clinician was paged to evaluate the patient. Our primary outcome measure was bundle compliance; our secondary measure was PT alert response compliance.

RESULTS

Baseline severe sepsis treatment bundle compliance was 37%. After requiring nursing response to the NT alert in 2016 and implementing the PT alert in 2018, our bundle compliance rose to 69% in 2020, demonstrating statistically significant difference (P = .006). PT alert response compliance rose from 67% in 2018 to 91% in 2020.

CONCLUSIONS

An interruptive severe sepsis screening alert sent directly to clinicians is a valuable tool to ensure prompt severe sepsis recognition and treatment. This biphasic alert system facilitated multidisciplinary collaboration in early sepsis diagnosis and management.

Automated digital technologies for supporting sepsis prediction in children: a scoping review protocol

INTRODUCTION

While there have been several literature reviews on the performance of digital sepsis prediction technologies and clinical decision-support algorithms for adults, there remains a knowledge gap in examining the development of automated technologies for sepsis prediction in children. This scoping review will critically analyse the current evidence on the design and performance of automated digital technologies to predict paediatric sepsis, to advance their development and integration within clinical settings.

METHODS AND ANALYSIS

This scoping review will follow Arksey and O'Malley's framework, conducted between February and December 2022. We will further develop the protocol using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for scoping reviews. We plan to search the following databases: Association of Computing Machinery (ACM) Digital Library, Cumulative Index to Nursing and Allied Health Literature (CINAHL), Embase, Google Scholar, Institute of Electric and Electronic Engineers (IEEE), PubMed, Scopus and Web of Science. Studies will be included on children >90 days postnatal to <21 years old, predicted to have or be at risk of developing sepsis by a digitalised model or algorithm designed for a clinical setting. Two independent reviewers will complete the abstract and full-text screening and the data extraction. Thematic analysis will be used to develop overarching concepts and present the narrative findings with quantitative results and descriptive statistics displayed in data tables.

ETHICS AND DISSEMINATION

Ethics approval for this scoping review study of the available literature is not required. We anticipate that the scoping review will identify the current evidence and design characteristics of digital prediction technologies for the timely and accurate prediction of paediatric sepsis and factors influencing clinical integration. We plan to disseminate the preliminary findings from this review at national and international research conferences in global and digital health, gathering critical feedback from multidisciplinary stakeholders. SCOPING REVIEW REGISTRATION: https://osf.io/veqha/?view_only=f560d4892d7c459ea4cff6dcdfacb086.

Computerized Clinical Decision Support Systems for the Early Detection of Sepsis Among Pediatric, Neonatal, and Maternal Inpatients: Scoping Review

BACKGROUND

Sepsis is a severe condition associated with extensive morbidity and mortality worldwide. Pediatric, neonatal, and maternal patients represent a considerable proportion of the sepsis burden. Identifying sepsis cases as early as possible is a key pillar of sepsis management and has prompted the development of sepsis identification rules and algorithms that are embedded in computerized clinical decision support (CCDS) systems.

OBJECTIVE

This scoping review aimed to systematically describe studies reporting on the use and evaluation of CCDS systems for the early detection of pediatric, neonatal, and maternal inpatients at risk of sepsis.

METHODS

MEDLINE, Embase, CINAHL, Cochrane, Latin American and Caribbean Health Sciences Literature (LILACS), Scopus, Web of Science, OpenGrey, ClinicalTrials.gov, and ProQuest Dissertations and Theses Global (PQDT) were searched by using a search strategy that incorporated terms for sepsis, clinical decision support, and early detection. Title, abstract, and full-text screening was performed by 2 independent reviewers, who consulted a third reviewer as needed. One reviewer performed data charting with a sample of data. This was checked by a second reviewer and via discussions with the review team, as necessary.

RESULTS

A total of 33 studies were included in this review-13 (39%) pediatric studies, 18 (55%) neonatal studies, and 2 (6%) maternal studies. All studies were published after 2011, and 27 (82%) were published from 2017 onward. The most common outcome investigated in pediatric studies was the accuracy of sepsis identification (9/13, 69%). Pediatric CCDS systems used different combinations of 18 diverse clinical criteria to detect sepsis across the 13 identified studies. In neonatal studies, 78% (14/18) of the studies investigated the Kaiser Permanente early-onset sepsis risk calculator. All studies investigated sepsis treatment and management outcomes, with 83% (15/18) reporting on antibiotics-related outcomes. Usability and cost-related outcomes were each reported in only 2 (6%) of the 31 pediatric or neonatal studies. Both studies on maternal populations were short abstracts.

CONCLUSIONS

This review found limited research investigating CCDS systems to support the early detection of sepsis among pediatric, neonatal, and maternal patients, despite the high burden of sepsis in these vulnerable populations. We have highlighted the need for a consensus definition for pediatric and neonatal sepsis and the study of usability and cost-related outcomes as critical areas for future research.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/24899.

Computerized Clinical Decision Support Systems for the Early Detection of Sepsis Among Adult Inpatients: Scoping Review

BACKGROUND

Sepsis is a significant cause of morbidity and mortality worldwide. Early detection of sepsis followed promptly by treatment initiation improves patient outcomes and saves lives. Hospitals are increasingly using computerized clinical decision support (CCDS) systems for the rapid identification of adult patients with sepsis.

OBJECTIVE

This scoping review aims to systematically describe studies reporting on the use and evaluation of CCDS systems for the early detection of adult inpatients with sepsis.

METHODS

The protocol for this scoping review was previously published. A total of 10 electronic databases (MEDLINE, Embase, CINAHL, the Cochrane database, LILACS [Latin American and Caribbean Health Sciences Literature], Scopus, Web of Science, OpenGrey, ClinicalTrials.gov, and PQDT [ProQuest Dissertations and Theses]) were comprehensively searched using terms for sepsis, CCDS, and detection to identify relevant studies. Title, abstract, and full-text screening were performed by 2 independent reviewers using predefined eligibility criteria. Data charting was performed by 1 reviewer with a second reviewer checking a random sample of studies. Any disagreements were discussed with input from a third reviewer. In this review, we present the results for adult inpatients, including studies that do not specify patient age.

RESULTS

A search of the electronic databases retrieved 12,139 studies following duplicate removal. We identified 124 studies for inclusion after title, abstract, full-text screening, and hand searching were complete. Nearly all studies (121/124, 97.6%) were published after 2009. Half of the studies were journal articles (65/124, 52.4%), and the remainder were conference abstracts (54/124, 43.5%) and theses (5/124, 4%). Most studies used a single cohort (54/124, 43.5%) or before-after (42/124, 33.9%) approach. Across all 124 included studies, patient outcomes were the most frequently reported outcomes (107/124, 86.3%), followed by sepsis treatment and management (75/124, 60.5%), CCDS usability (14/124, 11.3%), and cost outcomes (9/124, 7.3%). For sepsis identification, the systemic inflammatory response syndrome criteria were the most commonly used, alone (50/124, 40.3%), combined with organ dysfunction (28/124, 22.6%), or combined with other criteria (23/124, 18.5%). Over half of the CCDS systems (68/124, 54.8%) were implemented alongside other sepsis-related interventions.

CONCLUSIONS

The current body of literature investigating the implementation of CCDS systems for the early detection of adult inpatients with sepsis is extremely diverse. There is substantial variability in study design, CCDS criteria and characteristics, and outcomes measured across the identified literature. Future research on CCDS system usability, cost, and impact on sepsis morbidity is needed.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/24899.