1
|
van Rooden SM, van der Werff SD, van Mourik MSM, Lomholt F, Møller KL, Valk S, Dos Santos Ribeiro C, Wong A, Haitjema S, Behnke M, Rinaldi E. Federated systems for automated infection surveillance: a perspective. Antimicrob Resist Infect Control 2024; 13:113. [PMID: 39334278 PMCID: PMC11438042 DOI: 10.1186/s13756-024-01464-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/29/2024] [Accepted: 09/08/2024] [Indexed: 09/30/2024] Open
Abstract
Automation of surveillance of infectious diseases-where algorithms are applied to routine care data to replace manual decisions-likely reduces workload and improves quality of surveillance. However, various barriers limit large-scale implementation of automated surveillance (AS). Current implementation strategies for AS in surveillance networks include central implementation (i.e. collecting all data centrally, and central algorithm application for case ascertainment) or local implementation (i.e. local algorithm application and sharing surveillance results with the network coordinating center). In this perspective, we explore whether current challenges can be solved by federated AS. In federated AS, scripts for analyses are developed centrally and applied locally. We focus on the potential of federated AS in the context of healthcare associated infections (AS-HAI) and of severe acute respiratory illness (AS-SARI). AS-HAI and AS-SARI have common and specific requirements, but both would benefit from decreased local surveillance burden, alignment of AS and increased central and local oversight, and improved access to data while preserving privacy. Federated AS combines some benefits of a centrally implemented system, such as standardization and alignment of an easily scalable methodology, with some of the benefits of a locally implemented system including (near) real-time access to data and flexibility in algorithms, meeting different information needs and improving sustainability, and allowance of a broader range of clinically relevant case-definitions. From a global perspective, it can promote the development of automated surveillance where it is not currently possible and foster international collaboration.The necessary transformation of source data likely will place a significant burden on healthcare facilities. However, this may be outweighed by the potential benefits: improved comparability of surveillance results, flexibility and reuse of data for multiple purposes. Governance and stakeholder agreement to address accuracy, accountability, transparency, digital literacy, and data protection, warrants clear attention to create acceptance of the methodology. In conclusion, federated automated surveillance seems a potential solution for current barriers of large-scale implementation of AS-HAI and AS-SARI. Prerequisites for successful implementation include validation of results and evaluation requirements of network participants to govern understanding and acceptance of the methodology.
Collapse
Affiliation(s)
- Stephanie M van Rooden
- Department of Epidemiology and Surveillance, Centre for Infectious Disease Epidemiology and Surveillance, National Institute for Public Health and the Environment (RIVM), Bilthoven, The Netherlands.
| | - Suzanne D van der Werff
- Division of Infectious Diseases, Department of Medicine Solna, Karolinska Institutet, Stockholm, Sweden
- Department of Infectious Diseases, Karolinska University Healthcare Facility, Stockholm, Sweden
| | - Maaike S M van Mourik
- Department of Medical Microbiology and Infection Control, University Medical Centre Utrecht, Utrecht, The Netherlands
| | - Frederikke Lomholt
- Infectious Disease Epidemiology and Prevention, Statens Serum Institut, Copenhagen, Denmark
| | | | - Sarah Valk
- Department of Epidemiology and Surveillance, Centre for Infectious Disease Epidemiology and Surveillance, National Institute for Public Health and the Environment (RIVM), Bilthoven, The Netherlands
| | - Carolina Dos Santos Ribeiro
- Center for Infectious Disease Control, National Institute for Public Health and the Environment (RIVM), Bilthoven, The Netherlands
| | - Albert Wong
- Department of Statistics Data Science en Modelling, National Institute for Public Health and the Environment (RIVM), Bilthoven, The Netherlands
| | - Saskia Haitjema
- Central Diagnostic Laboratory, University Medical Centre Utrecht, Utrecht, The Netherlands
| | - Michael Behnke
- Institute of Hygiene and Environmental Medicine, Charité Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin and, Berlin Institute of Health, Berlin, Germany
- National Reference Center for the Surveillance of Nosocomial Infections, Charité Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin and Berlin Institute of Health, Berlin, Germany
| | - Eugenia Rinaldi
- Core Unit Digital Medicine and Interoperability, Berlin, Institute of Health at Charité - Universitätsmedizin Berlin, Berlin, Germany
| |
Collapse
|
2
|
Verberk JDM, van der Werff SD, Weegar R, Henriksson A, Richir MC, Buchli C, van Mourik MSM, Nauclér P. The augmented value of using clinical notes in semi-automated surveillance of deep surgical site infections after colorectal surgery. Antimicrob Resist Infect Control 2023; 12:117. [PMID: 37884948 PMCID: PMC10604406 DOI: 10.1186/s13756-023-01316-x] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2023] [Accepted: 09/25/2023] [Indexed: 10/28/2023] Open
Abstract
BACKGROUND In patients who underwent colorectal surgery, an existing semi-automated surveillance algorithm based on structured data achieves high sensitivity in detecting deep surgical site infections (SSI), however, generates a significant number of false positives. The inclusion of unstructured, clinical narratives to the algorithm may decrease the number of patients requiring manual chart review. The aim of this study was to investigate the performance of this semi-automated surveillance algorithm augmented with a natural language processing (NLP) component to improve positive predictive value (PPV) and thus workload reduction (WR). METHODS Retrospective, observational cohort study in patients who underwent colorectal surgery from January 1, 2015, through September 30, 2020. NLP was used to detect keyword counts in clinical notes. Several NLP-algorithms were developed with different count input types and classifiers, and added as component to the original semi-automated algorithm. Traditional manual surveillance was compared with the NLP-augmented surveillance algorithms and sensitivity, specificity, PPV and WR were calculated. RESULTS From the NLP-augmented models, the decision tree models with discretized counts or binary counts had the best performance (sensitivity 95.1% (95%CI 83.5-99.4%), WR 60.9%) and improved PPV and WR by only 2.6% and 3.6%, respectively, compared to the original algorithm. CONCLUSIONS The addition of an NLP component to the existing algorithm had modest effect on WR (decrease of 1.4-12.5%), at the cost of sensitivity. For future implementation it will be a trade-off between optimal case-finding techniques versus practical considerations such as acceptability and availability of resources.
Collapse
Affiliation(s)
- Janneke D M Verberk
- Department of Medical Microbiology and Infection Prevention, University Medical Centre Utrecht, Utrecht, the Netherlands
- Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht, the Netherlands
- Department of Epidemiology and Surveillance, Centre for Infectious Diseases Control, National Institute for Public Health and the Environment, Bilthoven, the Netherlands
| | - Suzanne D van der Werff
- Department of Medicine Solna, Division of Infectious Diseases, Karolinska Institutet, Stockholm, Sweden.
- Department of Infectious Diseases, Karolinska University Hospital, Stockholm, Sweden.
| | - Rebecka Weegar
- Department of Computer and Systems Sciences, Stockholm University, Stockholm, Sweden
| | - Aron Henriksson
- Department of Computer and Systems Sciences, Stockholm University, Stockholm, Sweden
| | - Milan C Richir
- Department of Surgery, Cancer Centre, University Medical Centre Utrecht, Utrecht, the Netherlands
| | - Christian Buchli
- Department of Molecular Medicine and Surgery, Karolinska Institutet, Stockholm, Sweden
- Department of Pelvic Cancer, GI Oncology and Colorectal Surgery Unit, Karolinska University Hospital, Stockholm, Sweden
| | - Maaike S M van Mourik
- Department of Medical Microbiology and Infection Prevention, University Medical Centre Utrecht, Utrecht, the Netherlands
| | - Pontus Nauclér
- Department of Medicine Solna, Division of Infectious Diseases, Karolinska Institutet, Stockholm, Sweden
- Department of Infectious Diseases, Karolinska University Hospital, Stockholm, Sweden
| |
Collapse
|
3
|
van der Werff SD, Verberk JDM, Buchli C, van Mourik MSM, Nauclér P. External validation of semi-automated surveillance algorithms for deep surgical site infections after colorectal surgery in an independent country. Antimicrob Resist Infect Control 2023; 12:96. [PMID: 37679824 PMCID: PMC10485951 DOI: 10.1186/s13756-023-01288-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2022] [Accepted: 08/12/2023] [Indexed: 09/09/2023] Open
Abstract
BACKGROUND Automated surveillance methods that re-use electronic health record data are considered an attractive alternative to traditional manual surveillance. However, surveillance algorithms need to be thoroughly validated before being implemented in a clinical setting. With semi-automated surveillance patients are classified as low or high probability of having developed infection, and only high probability patients subsequently undergo manual record review. The aim of this study was to externally validate two existing semi-automated surveillance algorithms for deep SSI after colorectal surgery, developed on Spanish and Dutch data, in a Swedish setting. METHODS The algorithms were validated in 225 randomly selected surgeries from Karolinska University Hospital from the period January 1, 2015 until August 31, 2020. Both algorithms were based on (re)admission and discharge data, mortality, reoperations, radiology orders, and antibiotic prescriptions, while one additionally used microbiology cultures. SSI was based on ECDC definitions. Sensitivity, specificity, positive predictive value, negative predictive value, and workload reduction were assessed compared to manual surveillance. RESULTS Both algorithms performed well, yet the algorithm not relying on microbiological culture data had highest sensitivity (97.6, 95%CI: 87.4-99.6), which was comparable to previously published results. The latter algorithm aligned best with clinical practice and would lead to 57% records less to review. CONCLUSIONS The results highlight the importance of thorough validation before implementation in other clinical settings than in which algorithms were originally developed: the algorithm excluding microbiology cultures had highest sensitivity in this new setting and has the potential to support large-scale semi-automated surveillance of SSI after colorectal surgery.
Collapse
Affiliation(s)
- Suzanne D van der Werff
- Department of Medicine Solna, Division of Infectious Diseases, Karolinska Institutet, Stockholm, 171 77, Sweden.
- Department of Infectious Diseases, Karolinska University Hospital, Stockholm, Sweden.
| | - Janneke D M Verberk
- Department of Medical Microbiology and Infection Prevention, University Medical Centre Utrecht, Utrecht, the Netherlands
- Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht, the Netherlands
- Department of Epidemiology and Surveillance, Centre for Infectious Diseases Control, National Institute for Public Health and the Environment, Bilthoven, the Netherlands
| | - Christian Buchli
- Department of Molecular Medicine and Surgery, Karolinska Institutet, Stockholm, Sweden
- Department of Pelvic Cancer, GI Oncology and Colorectal Surgery Unit, Karolinska University Hospital, Stockholm, Sweden
| | - Maaike S M van Mourik
- Department of Medical Microbiology and Infection Prevention, University Medical Centre Utrecht, Utrecht, the Netherlands
| | - Pontus Nauclér
- Department of Medicine Solna, Division of Infectious Diseases, Karolinska Institutet, Stockholm, 171 77, Sweden
- Department of Infectious Diseases, Karolinska University Hospital, Stockholm, Sweden
| |
Collapse
|
4
|
Verberk JDM, van der Kooi TII, Hetem DJ, Oostdam NEWM, Noordergraaf M, de Greeff SC, Bonten MJM, van Mourik MSM. Semiautomated surveillance of deep surgical site infections after colorectal surgeries: A multicenter external validation of two surveillance algorithms - ERRATUM. Infect Control Hosp Epidemiol 2023; 44:1208. [PMID: 36073094 PMCID: PMC10369218 DOI: 10.1017/ice.2022.213] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|