1
|
Zhang C, Wiens MO, Dunsmuir D, Pillay Y, Huxford C, Kimutai D, Tenywa E, Ouma M, Kigo J, Kamau S, Chege M, Kenya-Mugisha N, Mwaka S, Dumont GA, Kissoon N, Akech S, Ansermino JM. Geographical validation of the Smart Triage Model by age group. PLOS DIGITAL HEALTH 2024; 3:e0000311. [PMID: 38949998 PMCID: PMC11216563 DOI: 10.1371/journal.pdig.0000311] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/27/2023] [Accepted: 05/25/2024] [Indexed: 07/03/2024]
Abstract
Infectious diseases in neonates account for half of the under-five mortality in low- and middle-income countries. Data-driven algorithms such as clinical prediction models can be used to efficiently detect critically ill children in order to optimize care and reduce mortality. Thus far, only a handful of prediction models have been externally validated and are limited to neonatal in-hospital mortality. The aim of this study is to externally validate a previously derived clinical prediction model (Smart Triage) using a combined prospective baseline cohort from Uganda and Kenya with a composite endpoint of hospital admission, mortality, and readmission. We evaluated model discrimination using area under the receiver-operator curve (AUROC) and visualized calibration plots with age subsets (< 30 days, ≤ 2 months, ≤ 6 months, and < 5 years). Due to reduced performance in neonates (< 1 month), we re-estimated the intercept and coefficients and selected new thresholds to maximize sensitivity and specificity. 11595 participants under the age of five (under-5) were included in the analysis. The proportion with an endpoint ranged from 8.9% in all children under-5 (including neonates) to 26% in the neonatal subset alone. The model achieved good discrimination for children under-5 with AUROC of 0.81 (95% CI: 0.79-0.82) but poor discrimination for neonates with AUROC of 0.62 (95% CI: 0.55-0.70). Sensitivity at the low-risk thresholds (CI) were 85% (83%-87%) and 68% (58%-76%) for children under-5 and neonates, respectively. After model revision for neonates, we achieved an AUROC of 0.83 (95% CI: 0.79-0.87) with 13% and 41% as the low- and high-risk thresholds, respectively. The updated Smart Triage performs well in its predictive ability across different age groups and can be incorporated into current triage guidelines at local healthcare facilities. Additional validation of the model is indicated, especially for the neonatal model.
Collapse
Affiliation(s)
- Cherri Zhang
- Institute for Global Health, BC Children’s and Women’s Hospitals, Vancouver, British Columbia, Canada
| | - Matthew O. Wiens
- Institute for Global Health, BC Children’s and Women’s Hospitals, Vancouver, British Columbia, Canada
- Department of Anaesthesiology, Pharmacology & Therapeutics, University of British Columbia, Vancouver, British Columbia, Canada
- BC Children’s Hospital Research Institute, Vancouver, British Columbia, Canada
| | - Dustin Dunsmuir
- Institute for Global Health, BC Children’s and Women’s Hospitals, Vancouver, British Columbia, Canada
- BC Children’s Hospital Research Institute, Vancouver, British Columbia, Canada
| | - Yashodani Pillay
- Institute for Global Health, BC Children’s and Women’s Hospitals, Vancouver, British Columbia, Canada
| | - Charly Huxford
- Institute for Global Health, BC Children’s and Women’s Hospitals, Vancouver, British Columbia, Canada
| | | | | | - Mary Ouma
- Mbagathi County Hospital, Nairobi, Kenya
| | - Joyce Kigo
- Health Services Unit, KEMRI-Wellcome Trust Research Program, Nairobi, Kenya
| | - Stephen Kamau
- Health Services Unit, KEMRI-Wellcome Trust Research Program, Nairobi, Kenya
| | - Mary Chege
- Department of Pediatrics, Kiambu County Referral Hospital, Kiambu, Kenya
| | | | - Savio Mwaka
- World Alliance for Lung and Intensive Care Medicine in Uganda, Kampala, Uganda
| | - Guy A. Dumont
- Department of Anaesthesiology, Pharmacology & Therapeutics, University of British Columbia, Vancouver, British Columbia, Canada
- BC Children’s Hospital Research Institute, Vancouver, British Columbia, Canada
- Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, British Columbia, Canada
| | - Niranjan Kissoon
- BC Children’s Hospital Research Institute, Vancouver, British Columbia, Canada
- Department of Pediatrics, University of British Columbia, Vancouver, British Columbia, Canada
| | - Samuel Akech
- Health Services Unit, KEMRI-Wellcome Trust Research Program, Nairobi, Kenya
| | - J Mark Ansermino
- Institute for Global Health, BC Children’s and Women’s Hospitals, Vancouver, British Columbia, Canada
- Department of Anaesthesiology, Pharmacology & Therapeutics, University of British Columbia, Vancouver, British Columbia, Canada
- BC Children’s Hospital Research Institute, Vancouver, British Columbia, Canada
| | | |
Collapse
|
2
|
Mawji A, Li E, Dunsmuir D, Komugisha C, Novakowski SK, Wiens MO, Vesuvius TA, Kissoon N, Ansermino JM. Smart triage: Development of a rapid pediatric triage algorithm for use in low-and-middle income countries. Front Pediatr 2022; 10:976870. [PMID: 36483471 PMCID: PMC9723221 DOI: 10.3389/fped.2022.976870] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/23/2022] [Accepted: 11/01/2022] [Indexed: 11/23/2022] Open
Abstract
INTRODUCTION Early and accurate recognition of children at risk of progressing to critical illness could contribute to improved patient outcomes and resource allocation. In resource limited settings digital triage tools can support decision making and improve healthcare delivery. We developed a model for rapid identification of critically ill children at triage. METHODS This was a prospective cohort study of acutely ill children presenting at Jinja Regional Referral Hospital in Eastern Uganda. Variables collected in the emergency department informed the development of a logistic model based on hospital admission using bootstrap stepwise regression. Low and high-risk thresholds for 90% minimum sensitivity and specificity, respectively generated three risk level categories. Performance was assessed using receiver operating characteristic curve analysis on a held-out test set generated by an 80:20 split with 10-fold cross validation. A risk stratification table informed clinical interpretation. RESULTS The model derivation cohort included 1,612 participants, with an admission rate of approximately 23%. The majority of admitted patients were under five years old and presenting with sepsis, malaria, or pneumonia. A 9-predictor triage model was derived: logit (p) = -32.888 + (0.252, square root of age) + (0.016, heart rate) + (0.819, temperature) + (-0.022, mid-upper arm circumference) + (0.048 transformed oxygen saturation) + (1.793, parent concern) + (1.012, difficulty breathing) + (1.814, oedema) + (1.506, pallor). The model afforded good discrimination, calibration, and risk stratification at the selected thresholds of 8% and 40%. CONCLUSION In a low income, pediatric population, we developed a nine variable triage model with high sensitivity and specificity to predict who should be admitted. The triage model can be integrated into any digital platform and used with minimal training to guide rapid identification of critically ill children at first contact. External validation and clinical implementation are in progress.
Collapse
Affiliation(s)
- Alishah Mawji
- Department of Anesthesiology, Pharmacology & Therapeutics, University of British Columbia, Vancouver, BC, Canada.,Centre for International Child Health, BC Children's Hospital Research Institute, Vancouver, BC, Canada
| | - Edmond Li
- Department of Anesthesiology, Pharmacology & Therapeutics, University of British Columbia, Vancouver, BC, Canada
| | - Dustin Dunsmuir
- Department of Anesthesiology, Pharmacology & Therapeutics, University of British Columbia, Vancouver, BC, Canada.,Centre for International Child Health, BC Children's Hospital Research Institute, Vancouver, BC, Canada
| | | | - Stefanie K Novakowski
- Department of Anesthesiology, Pharmacology & Therapeutics, University of British Columbia, Vancouver, BC, Canada.,Centre for International Child Health, BC Children's Hospital Research Institute, Vancouver, BC, Canada
| | - Matthew O Wiens
- Centre for International Child Health, BC Children's Hospital Research Institute, Vancouver, BC, Canada
| | | | - Niranjan Kissoon
- Department of Pediatrics, University of British Columbia, Vancouver, BC, Canada
| | - J Mark Ansermino
- Department of Anesthesiology, Pharmacology & Therapeutics, University of British Columbia, Vancouver, BC, Canada.,Centre for International Child Health, BC Children's Hospital Research Institute, Vancouver, BC, Canada
| |
Collapse
|
3
|
Mawji A, Li E, Komugisha C, Akech S, Dunsmuir D, Wiens MO, Kissoon N, Kenya-Mugisha N, Tagoola A, Kimutai D, Bone JN, Dumont G, Ansermino JM. Smart triage: triage and management of sepsis in children using the point-of-care Pediatric Rapid Sepsis Trigger (PRST) tool. BMC Health Serv Res 2020; 20:493. [PMID: 32493319 PMCID: PMC7268489 DOI: 10.1186/s12913-020-05344-w] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2020] [Accepted: 05/20/2020] [Indexed: 11/25/2022] Open
Abstract
BACKGROUND Sepsis is the leading cause of death and disability in children. Every hour of delay in treatment is associated with an escalating risk of morbidity and mortality. The burden of sepsis is greatest in low- and middle-income countries where timely treatment may not occur due to delays in diagnosis and prioritization of critically ill children. To circumvent these challenges, we propose the development and clinical evaluation of a digital triage tool that will identify high risk children and reduce time to treatment. We will also implement and clinically validate a Radio-Frequency Identification system to automate tracking of patients. The mobile platform (mobile device and dashboard) and automated patient tracking system will create a low cost, highly scalable solution for critically ill children, including those with sepsis. METHODS This is pre-post intervention study consisting of three phases. Phase I will be a baseline period where data is collected on key predictors and outcomes before implementation of the digital triage tool. In Phase I, there will be no changes to healthcare delivery processes in place at the study hospitals. Phase II will involve model derivation, technology development, and usability testing. Phase III will be the intervention period where data is collected on key predictors and outcomes after implementation of the digital triage tool. The primary outcome, time to treatment initiation, will be compared to assess effectiveness of the digital health intervention. DISCUSSION Smart technology has the potential to overcome the barrier of limited clinical expertise in the identification of the child at risk. This mobile health platform, with sensors and data-driven applications, will provide real-time individualized risk prediction to rapidly triage patients and facilitate timely access to life-saving treatments for children in low- and middle-income countries, where specialists are not regularly available and deaths from sepsis are common. TRIAL REGISTRATION Clinical Trials.gov Identifier: NCT04304235, Registered 11 March 2020.
Collapse
Affiliation(s)
- Alishah Mawji
- Department of Anesthesiology, Pharmacology & Therapeutics, University of British Columbia, 217-2176 Health Sciences Mall, Vancouver, BC, V6T 1Z3, Canada.
| | - Edmond Li
- School of Population and Public Health, University of British Columbia, 2206 East Mall, Vancouver, BC, V6T 1Z3, Canada
| | - Clare Komugisha
- Walimu, P.O. Box 9924, Plot 5-7, Coral Crescent, Kololo, Kampala, Uganda
| | - Samuel Akech
- Kenya Medical Research Institute/Wellcome Trust Research Programme, P.O. Box 43640-00100, Nairobi, Kenya
| | - Dustin Dunsmuir
- Digital Health Innovation Lab, BC Children's Hospital Research Institute, 948 W 28th Ave, Vancouver, BC, V5Z 4H4, Canada
| | - Matthew O Wiens
- Center for International Child Health, BC Children's Hospital Research Institute, 948 W 28th Ave, Vancouver, BC, V5Z 4H4, Canada
| | - Niranjan Kissoon
- Department of Pediatrics, University of British Columbia, Rm B2W, 4480 Oak Street, Vancouver, BC, V6H 3V4, Canada
| | | | | | - David Kimutai
- Mbagathi County Hospital, P.O. Box 20725-00202, Nairobi, Kenya
| | - Jeffrey N Bone
- Department of Obstetrics and Gynaecology, University of British Columbia, 1125 Howe Street, Vancouver, BC, V6Z 2K8, Canada
| | - Guy Dumont
- Electrical and Computer Engineering, The University of British Columbia, 5500 - 2332 Main Mall, Vancouver, BC, V6T 1Z4, Canada
| | - J Mark Ansermino
- Department of Anesthesiology, Pharmacology & Therapeutics, University of British Columbia, 217-2176 Health Sciences Mall, Vancouver, BC, V6T 1Z3, Canada
| |
Collapse
|