Semiautomated surveillance of deep surgical site infections after colorectal surgeries: A multicenter external validation of two surveillance algorithms

Federated systems for automated infection surveillance: a perspective

Automation of surveillance of infectious diseases-where algorithms are applied to routine care data to replace manual decisions-likely reduces workload and improves quality of surveillance. However, various barriers limit large-scale implementation of automated surveillance (AS). Current implementation strategies for AS in surveillance networks include central implementation (i.e. collecting all data centrally, and central algorithm application for case ascertainment) or local implementation (i.e. local algorithm application and sharing surveillance results with the network coordinating center). In this perspective, we explore whether current challenges can be solved by federated AS. In federated AS, scripts for analyses are developed centrally and applied locally. We focus on the potential of federated AS in the context of healthcare associated infections (AS-HAI) and of severe acute respiratory illness (AS-SARI). AS-HAI and AS-SARI have common and specific requirements, but both would benefit from decreased local surveillance burden, alignment of AS and increased central and local oversight, and improved access to data while preserving privacy. Federated AS combines some benefits of a centrally implemented system, such as standardization and alignment of an easily scalable methodology, with some of the benefits of a locally implemented system including (near) real-time access to data and flexibility in algorithms, meeting different information needs and improving sustainability, and allowance of a broader range of clinically relevant case-definitions. From a global perspective, it can promote the development of automated surveillance where it is not currently possible and foster international collaboration.The necessary transformation of source data likely will place a significant burden on healthcare facilities. However, this may be outweighed by the potential benefits: improved comparability of surveillance results, flexibility and reuse of data for multiple purposes. Governance and stakeholder agreement to address accuracy, accountability, transparency, digital literacy, and data protection, warrants clear attention to create acceptance of the methodology. In conclusion, federated automated surveillance seems a potential solution for current barriers of large-scale implementation of AS-HAI and AS-SARI. Prerequisites for successful implementation include validation of results and evaluation requirements of network participants to govern understanding and acceptance of the methodology.

The augmented value of using clinical notes in semi-automated surveillance of deep surgical site infections after colorectal surgery

BACKGROUND

In patients who underwent colorectal surgery, an existing semi-automated surveillance algorithm based on structured data achieves high sensitivity in detecting deep surgical site infections (SSI), however, generates a significant number of false positives. The inclusion of unstructured, clinical narratives to the algorithm may decrease the number of patients requiring manual chart review. The aim of this study was to investigate the performance of this semi-automated surveillance algorithm augmented with a natural language processing (NLP) component to improve positive predictive value (PPV) and thus workload reduction (WR).

METHODS

Retrospective, observational cohort study in patients who underwent colorectal surgery from January 1, 2015, through September 30, 2020. NLP was used to detect keyword counts in clinical notes. Several NLP-algorithms were developed with different count input types and classifiers, and added as component to the original semi-automated algorithm. Traditional manual surveillance was compared with the NLP-augmented surveillance algorithms and sensitivity, specificity, PPV and WR were calculated.

RESULTS

From the NLP-augmented models, the decision tree models with discretized counts or binary counts had the best performance (sensitivity 95.1% (95%CI 83.5-99.4%), WR 60.9%) and improved PPV and WR by only 2.6% and 3.6%, respectively, compared to the original algorithm.

CONCLUSIONS

The addition of an NLP component to the existing algorithm had modest effect on WR (decrease of 1.4-12.5%), at the cost of sensitivity. For future implementation it will be a trade-off between optimal case-finding techniques versus practical considerations such as acceptability and availability of resources.

External validation of semi-automated surveillance algorithms for deep surgical site infections after colorectal surgery in an independent country

BACKGROUND

Automated surveillance methods that re-use electronic health record data are considered an attractive alternative to traditional manual surveillance. However, surveillance algorithms need to be thoroughly validated before being implemented in a clinical setting. With semi-automated surveillance patients are classified as low or high probability of having developed infection, and only high probability patients subsequently undergo manual record review. The aim of this study was to externally validate two existing semi-automated surveillance algorithms for deep SSI after colorectal surgery, developed on Spanish and Dutch data, in a Swedish setting.

METHODS

The algorithms were validated in 225 randomly selected surgeries from Karolinska University Hospital from the period January 1, 2015 until August 31, 2020. Both algorithms were based on (re)admission and discharge data, mortality, reoperations, radiology orders, and antibiotic prescriptions, while one additionally used microbiology cultures. SSI was based on ECDC definitions. Sensitivity, specificity, positive predictive value, negative predictive value, and workload reduction were assessed compared to manual surveillance.

RESULTS

Both algorithms performed well, yet the algorithm not relying on microbiological culture data had highest sensitivity (97.6, 95%CI: 87.4-99.6), which was comparable to previously published results. The latter algorithm aligned best with clinical practice and would lead to 57% records less to review.

CONCLUSIONS

The results highlight the importance of thorough validation before implementation in other clinical settings than in which algorithms were originally developed: the algorithm excluding microbiology cultures had highest sensitivity in this new setting and has the potential to support large-scale semi-automated surveillance of SSI after colorectal surgery.