Clinical prediction models to diagnose neonatal sepsis: a scoping review protocol

Development and Implementation of Digital Diagnostic Algorithms for Neonatal Units in Zimbabwe and Malawi: Development and Usability Study

BACKGROUND

Despite an increase in hospital-based deliveries, neonatal mortality remains high in low-resource settings. Due to limited laboratory diagnostics, there is significant reliance on clinical findings to inform diagnoses. Accurate, evidence-based identification and management of neonatal conditions could improve outcomes by standardizing care. This could be achieved through digital clinical decision support (CDS) tools. Neotree is a digital, quality improvement platform that incorporates CDS, aiming to improve neonatal care in low-resource health care facilities. Before this study, first-phase CDS development included developing and implementing neonatal resuscitation algorithms, creating initial versions of CDS to address a range of neonatal conditions, and a Delphi study to review key algorithms.

OBJECTIVE

This second-phase study aims to codevelop and implement neonatal digital CDS algorithms in Malawi and Zimbabwe.

METHODS

Overall, 11 diagnosis-specific web-based workshops with Zimbabwean, Malawian, and UK neonatal experts were conducted (August 2021 to April 2022) encompassing the following: (1) review of available evidence, (2) review of country-specific guidelines (Essential Medicines List and Standard Treatment Guidelinesfor Zimbabwe and Care of the Infant and Newborn, Malawi), and (3) identification of uncertainties within the literature for future studies. After agreement of clinical content, the algorithms were programmed into a test script, tested with the respective hospital's health care professionals (HCPs), and refined according to their feedback. Once finalized, the algorithms were programmed into the Neotree software and implemented at the tertiary-level implementation sites: Sally Mugabe Central Hospital in Zimbabwe and Kamuzu Central Hospital in Malawi, in December 2021 and May 2022, respectively. In Zimbabwe, usability was evaluated through 2 usability workshops and usability questionnaires: Post-Study System Usability Questionnaire (PSSUQ) and System Usability Scale (SUS).

RESULTS

Overall, 11 evidence-based diagnostic and management algorithms were tailored to local resource availability. These refined algorithms were then integrated into Neotree. Where national management guidelines differed, country-specific guidelines were created. In total, 9 HCPs attended the usability workshops and completed the SUS, among whom 8 (89%) completed the PSSUQ. Both usability scores (SUS mean score 75.8 out of 100 [higher score is better]; PSSUQ overall score 2.28 out of 7 [lower score is better]) demonstrated high usability of the CDS function but highlighted issues around technical complexity, which continue to be addressed iteratively.

CONCLUSIONS

This study describes the successful development and implementation of the only known neonatal CDS system, incorporated within a bedside data capture system with the ability to deliver up-to-date management guidelines, tailored to local resource availability. This study highlighted the importance of collaborative participatory design. Further implementation evaluation is planned to guide and inform the development of health system and program strategies to support newborn HCPs, with the ultimate goal of reducing preventable neonatal morbidity and mortality in low-resource settings.

Improving the Treatment of Neonatal Sepsis in Resource-Limited Settings: Gaps and Recommendations

Neonatal sepsis causes significant global morbidity and mortality, with the highest burden in resource-limited settings where 99% of neonatal deaths occur. There are multiple challenges to achieving successful treatment of neonates in this setting. Firstly, reliable and low-cost strategies for risk identification are urgently needed to facilitate treatment as early as possible. Improved laboratory capacity to allow identification of causative organisms would support antimicrobial stewardship. Antibiotic treatment is still hampered by availability, but also increasingly by antimicrobial resistance - making surveillance of organisms and judicious antibiotic use a priority. Finally, supportive care is key in the management of the neonate with sepsis and has been underrecognized as a priority in resource-limited settings. This includes fluid balance and nutritional support in the acute phase, and follow-up care in order to mitigate complications and optimise long-term outcomes. There is much more work to be done in identifying the holistic needs of neonates and their families to provide effective family-integrated interventions and complete the package of neonatal sepsis management in resource-limited settings.

Diagnosing early-onset neonatal sepsis in low-resource settings: development of a multivariable prediction model

OBJECTIVE

To develop a clinical prediction model to diagnose neonatal sepsis in low-resource settings.

DESIGN

Secondary analysis of data collected by the Neotree digital health system from 1 February 2019 to 31 March 2020. We used multivariable logistic regression with candidate predictors identified from expert opinion and literature review. Missing data were imputed using multivariate imputation and model performance was evaluated in the derivation cohort.

SETTING

A tertiary neonatal unit at Sally Mugabe Central Hospital, Zimbabwe.

PATIENTS

We included 2628 neonates aged <72 hours, gestation ≥32+0 weeks and birth weight ≥1500 g.

INTERVENTIONS

Participants received standard care as no specific interventions were dictated by the study protocol.

MAIN OUTCOME MEASURES

Clinical early-onset neonatal sepsis (within the first 72 hours of life), defined by the treating consultant neonatologist.

RESULTS

Clinical early-onset sepsis was diagnosed in 297 neonates (11%). The optimal model included eight predictors: maternal fever, offensive liquor, prolonged rupture of membranes, neonatal temperature, respiratory rate, activity, chest retractions and grunting. Receiver operating characteristic analysis gave an area under the curve of 0.74 (95% CI 0.70-0.77). For a sensitivity of 95% (92%-97%), corresponding specificity was 11% (10%-13%), positive predictive value 12% (11%-13%), negative predictive value 95% (92%-97%), positive likelihood ratio 1.1 (95% CI 1.0-1.1) and negative likelihood ratio 0.4 (95% CI 0.3-0.6).

CONCLUSIONS

Our clinical prediction model achieved high sensitivity with low specificity, suggesting it may be suited to excluding early-onset sepsis. Future work will validate and update this model before considering implementation within the Neotree.

Development and implementation experience of a learning healthcare system for facility based newborn care in low resource settings: The Neotree

Introduction

Improving peri- and postnatal facility-based care in low-resource settings (LRS) could save over 6000 babies' lives per day. Most of the annual 2.4 million neonatal deaths and 2 million stillbirths occur in healthcare facilities in LRS and are preventable through the implementation of cost-effective, simple, evidence-based interventions. However, their implementation is challenging in healthcare systems where one in four babies admitted to neonatal units die. In high-resource settings healthcare systems strengthening is increasingly delivered via learning healthcare systems to optimise care quality, but this approach is rare in LRS.

Methods

Since 2014 we have worked in Bangladesh, Malawi, Zimbabwe, and the UK to co-develop and pilot the Neotree system: an android application with accompanying data visualisation, linkage, and export. Its low-cost hardware and state-of-the-art software are used to support healthcare professionals to improve postnatal care at the bedside and to provide insights into population health trends. Here we summarise the formative conceptualisation, development, and preliminary implementation experience of the Neotree.

Results

Data thus far from ~18 000 babies, 400 healthcare professionals in four hospitals (two in Zimbabwe, two in Malawi) show high acceptability, feasibility, usability, and improvements in healthcare professionals' ability to deliver newborn care. The data also highlight gaps in knowledge in newborn care and quality improvement. Implementation has been resilient and informative during external crises, for example, coronavirus disease 2019 (COVID-19) pandemic. We have demonstrated evidence of improvements in clinical care and use of data for Quality Improvement (QI) projects.

Conclusion

Human-centred digital development of a QI system for newborn care has demonstrated the potential of a sustainable learning healthcare system to improve newborn care and outcomes in LRS. Pilot implementation evaluation is ongoing in three of the four aforementioned hospitals (two in Zimbabwe and one in Malawi) and a larger scale clinical cost effectiveness trial is planned.

Infectious aetiologies of neonatal illness in south Asia classified using WHO definitions: a primary analysis of the ANISA study

Background

Globally, neonatal mortality accounts for almost half of all deaths in children younger than 5 years. Aetiological agents of neonatal infection are difficult to identify because the clinical signs are non-specific. Using data from the Aetiology of Neonatal Infections in south Asia (ANISA) cohort, we aimed to describe the spectrum of infectious aetiologies of acute neonatal illness categorised post-hoc using the 2015 WHO case definitions of critical illness, clinical severe infection, and fast breathing only.

Methods

Eligible infants were aged 0–59 days with possible serious bacterial infection and healthy infants enrolled in the ANISA study in Bangladesh, India, and Pakistan. We applied a partial latent class Bayesian model to estimate the prevalence of 27 pathogens detectable on PCR, pathogens detected by blood culture only, and illness not attributed to any infectious aetiology. Infants with at least one clinical specimen available were included in the analysis. We assessed the prevalence of these aetiologies according to WHO's case definitions of critically ill, clinical severe infection, and infants with late onset, isolated fast breathing. For the clinical severe definition, we compared the prevalence of signs by bacterial versus viral aetiology.

Findings

There were 934 infants (992 episodes) in the critically ill category, 3769 (4000 episodes) in the clinical severe infection category, and 738 (771 episodes) in the late-onset isolated fast breathing category. We estimated the proportion of illness attributable to bacterial infection was 32·7% in infants in the critically ill group, 15·6% in the clinical severe infection group, and 8·8% among infants with late-onset isolated fast breathing group. An infectious aetiology was not identified in 58–82% of infants in these categories. Among 4000 episodes of clinical severe infection, those with bacterial versus viral attribution had higher proportions of hypothermia, movement only when stimulated, convulsions, and poor feeding.

Interpretation

Our modelled results generally support the revised WHO case definitions, although a revision of the most severe case definition could be considered. Clinical criteria do not clearly differentiate between young infants with and without infectious aetiologies. Our results highlight the need for improved point-of-care diagnostics, and further study into neonatal deaths and episodes with no identified aetiology, to ensure antibiotic stewardship and targeted interventions.

Funding

The Bill and Melinda Gates Foundation.

Predictive Scores for Late-Onset Neonatal Sepsis as an Early Diagnostic and Antimicrobial Stewardship Tool: What Have We Done So Far?

Background: Late-onset neonatal sepsis (LOS) represents a significant cause of morbidity and mortality worldwide, and early diagnosis remains a challenge. Various ‘sepsis scores’ have been developed to improve early identification. The aim of the current review is to summarize the current knowledge on the utility of predictive scores in LOS as a tool for early sepsis recognition, as well as an antimicrobial stewardship tool. Methods: The following research question was developed: Can we diagnose LOS with accuracy in neonates using a predictive score? A systematic search was performed in the PubMed database from 1982 (first predictive score published) to December 2021. Results: Some (1352) articles were identified—out of which, 16 were included in the review. Eight were original scores, five were validations of already existing scores and two were mixed. Predictive models were developed by combining a variety of clinical, laboratory and other variables. The majority were found to assist in early diagnosis, but almost all had a limited diagnostic accuracy. Conclusions: There is an increasing need worldwide for a simple and accurate score to promptly predict LOS. Combinations of the selected parameters may be helpful, but until now, a single score has not been proven to be comprehensive.

A data-driven health index for neonatal morbidities

Whereas prematurity is a major cause of neonatal mortality, morbidity, and lifelong impairment, the degree of prematurity is usually defined by the gestational age (GA) at delivery rather than by neonatal morbidity. Here we propose a multi-task deep neural network model that simultaneously predicts twelve neonatal morbidities, as the basis for a new data-driven approach to define prematurity. Maternal demographics, medical history, obstetrical complications, and prenatal fetal findings were obtained from linked birth certificates and maternal/infant hospitalization records for 11,594,786 livebirths in California from 1991 to 2012. Overall, our model outperformed traditional models to assess prematurity which are based on GA and/or birthweight (area under the precision-recall curve was 0.326 for our model, 0.229 for GA, and 0.156 for small for GA). These findings highlight the potential of using machine learning techniques to predict multiple prematurity phenotypes and inform clinical decisions to prevent, diagnose and treat neonatal morbidities.

•

Traditional definitions of prematurity based on gestational age need to be updated

•

Deep learning of maternal clinical data improves predictions of neonatal morbidity

•

Data-driven model leverages birthweight, type of delivery and maternal race

•

Accurate risk prediction can inform clinical decisions

Development and validation of a sepsis diagnostic scoring model for neonates with suspected sepsis

BACKGROUND

We aimed to develop and validate a diagnostic model for sepsis among neonates evaluated for suspected sepsis, by incorporating thromboelastometry parameters, maternal/neonatal risk factors, clinical signs/symptoms and laboratory results.

METHODS

This retrospective cohort study included 291 neonates with presumed sepsis, hospitalized in a NICU, from 07/2014 to 07/2021. Laboratory tests were obtained on disease onset and prior to initiating antibiotic therapy. Τhromboelastometry extrinsically activated (EXTEM) assay was performed simultaneously and Tοllner and nSOFA scores were calculated. Sepsis diagnosis was the outcome variable. A 10-fold cross-validation least absolute shrinkage and selection operator logit regression procedure was applied to derive the final multivariable score. Clinical utility was evaluated by decision curve analysis.

RESULTS

Gestational age, CRP, considerable skin discoloration, liver enlargement, neutrophil left shift, and EXTEM A10, were identified as the strongest predictors and included in the Neonatal Sepsis Diagnostic (NeoSeD) model. NeoSeD score demonstrated excellent discrimination capacity for sepsis and septic shock with an AUC: 0.918 (95% CI, 0.884-0.952) and 0.974 (95% CI, 0.958-0.989) respectively, which was significantly higher compared to Töllner and nSOFA scores.

CONCLUSIONS

The NeoSeD score is simple, accurate, practical, and may contribute to a timely diagnosis of sepsis in neonates with suspected sepsis. External validation in multinational cohorts is necessary before clinical application.

Evaluation of presepsin as a diagnostic tool in newborns with risk of early-onset neonatal sepsis

OBJECTIVES

To evaluate the efficacy of presepsin (P-SEP) as a potential biomarker of early-onset neonatal sepsis (EOS) and compare it to other routinely used markers of inflammation. To establish the cut-off values of P-SEP for EOS.

STUDY DESIGN

184 newborns were prospectively recruited between January 2018 to December 2020. Newborns >34th gestational week with suspected infection were included up to 72 h after delivery, and divided into three categories (i.e., unlikely, possible, and probable infection) based on risk factors, clinical symptoms and laboratory results. Values of plasma P-SEP were sequentially analyzed.

RESULTS

Median values of P-SEP in newborns with probable infection were significantly higher compared to healthy newborns (p = 0.0000013) and unlikely infection group (p = 0.0000025). The AUC for discriminating the probable infection group from the unlikely infection group was 0.845 (95% Cl: 0.708-0.921). The diagnostic efficacy of P-SEP was highest when used in combination with IL-6 and CRP (0.97; 95% CI: 0.911-0.990). The optimal cut-off value of P-SEP was determined to be 695 ng/L.

CONCLUSION

P-SEP, when combined with IL-6 and CRP, may be utilized as a negative predictive marker of EOS (NPV 97.2%, 95% CI: 93.3-101), especially in newborns at low to medium risk of infection.

Relevance of Biomarkers Currently in Use or Research for Practical Diagnosis Approach of Neonatal Early-Onset Sepsis

Neonatal early-onset sepsis (EOS) is defined as an invasive infection that occurs in the first 72 h of life. The incidence of EOS varies from 0.5–2% live births in developed countries, up to 9.8% live births in low resource settings, generating a high mortality rate, especially in extremely low birth weight neonates. Clinical signs are nonspecific, leading to a late diagnosis and high mortality. Currently, there are several markers used for sepsis evaluation, such as hematological indices, acute phase reactants, cytokines, which by themselves do not show acceptable sensitivity and specificity for the diagnosis of EOS in neonates. Newer and more selective markers have surfaced recently, such as presepsin and endocan, but they are currently only in the experimental research stages. This comprehensive review article is based on the role of biomarkers currently in use or in the research phase from a basic, translational, and clinical viewpoint that helps us to improve the quality of neonatal early-onset sepsis diagnosis and management.