Automated surveillance systems for healthcare-associated infections: results from a European survey and experiences from real-life utilization

Automating surgical procedure extraction for society of surgeons adult cardiac surgery registry using pretrained language models

Objective

Surgical registries play a crucial role in clinical knowledge discovery, hospital quality assurance, and quality improvement. However, maintaining a surgical registry requires significant monetary and human resources given the wide gamut of information abstracted from medical records ranging from patient co-morbidities to procedural details to post-operative outcomes. Although natural language processing (NLP) methods such as pretrained language models (PLMs) have promised automation of this process, there are yet substantial barriers to implementation. In particular, constant shifts in both underlying data and required registry content are hurdles to the application of NLP technologies.

Materials and Methods

In our work, we evaluate the application of PLMs for automating the population of the Society of Thoracic Surgeons (STSs) adult cardiac surgery registry (ACS) procedural elements, for which we term Cardiovascular Surgery Bidirectional Encoder Representations from Transformers (CS-BERT). CS-BERT was validated across multiple satellite sites and versions of the STS-ACS registry.

Results

CS-BERT performed well (F1 score of 0.8417 ± 0.1838) in common cardiac surgery procedures compared to models based on diagnosis codes (F1 score of 0.6130 ± 0.0010). The model also generalized well to satellite sites and across different versions of the STS-ACS registry.

Discussion and Conclusions

This study provides evidence that PLMs can be used to extract the more common cardiac surgery procedure variables in the STS-ACS registry, potentially reducing need for expensive human annotation and wide scale dissemination. Further research is needed for rare procedural variables which suffer from both lack of data and variable documentation quality.

Healthcare as a driver, reservoir and amplifier of antimicrobial resistance: opportunities for interventions

Antimicrobial resistance (AMR) is a global health challenge that threatens humans, animals and the environment. Evidence is emerging for a role of healthcare infrastructure, environments and patient pathways in promoting and maintaining AMR via direct and indirect mechanisms. Advances in vaccination and monoclonal antibody therapies together with integrated surveillance, rapid diagnostics, targeted antimicrobial therapy and infection control measures offer opportunities to address healthcare-associated AMR risks more effectively. Additionally, innovations in artificial intelligence, data linkage and intelligent systems can be used to better predict and reduce AMR and improve healthcare resilience. In this Review, we examine the mechanisms by which healthcare functions as a driver, reservoir and amplifier of AMR, contextualized within a One Health framework. We also explore the opportunities and innovative solutions that can be used to combat AMR throughout the patient journey. We provide a perspective on the current evidence for the effectiveness of interventions designed to mitigate healthcare-associated AMR and promote healthcare resilience within high-income and resource-limited settings, as well as the challenges associated with their implementation.

Comparing automated surveillance systems for detection of pathogen-related clusters in healthcare settings

BACKGROUND

Detection of pathogen-related clusters within a hospital is key to early intervention to prevent onward transmission. Various automated surveillance methods for outbreak detection have been implemented in hospital settings. However, direct comparison is difficult due to heterogenicity of data sources and methodologies. In the hospital setting, we assess the performance of three different methods for identifying microbiological clusters when applied to various pathogens with distinct occurrence patterns.

METHODS

In this retrospective cohort study we use WHONET-SaTScan, CLAR (CLuster AleRt system) and our currently used percentile-based system (P75) for the means of cluster detection. The three methods are applied to the same data curated from 1st January 2014 to 31st December 2021 from a tertiary care hospital. We show the results for the following case studies: the introduction of a new pathogen with subsequent endemicity, an endemic species, rising levels of an endemic organism, and a sporadically occurring species.

RESULTS

All three cluster detection methods showed congruence only in endemic organisms. However, there was a paucity of alerts from WHONET-SaTScan (n = 9) compared to CLAR (n = 319) and the P75 system (n = 472). WHONET-SaTScan did not pick up smaller variations in baseline numbers of endemic organisms as well as sporadic organisms as compared to CLAR and the P75 system. CLAR and the P75 system revealed congruence in alerts for both endemic and sporadic organisms.

CONCLUSIONS

Use of statistically based automated cluster alert systems (such as CLAR and WHONET-Satscan) are comparable to rule-based alert systems only for endemic pathogens. For sporadic pathogens WHONET-SaTScan returned fewer alerts compared to rule-based alert systems. Further work is required regarding clinical relevance, timelines of cluster alerts and implementation.

Evaluation of a multifaceted implementation strategy for semi-automated surveillance of surgical site infections after total hip or knee arthroplasty: a multicentre pilot study in the Netherlands

INTRODUCTION

To promote the nation-wide implementation of semi-automated surveillance (AS) of surgical site infection after hip and knee arthroplasty, the Dutch National Institute for Public Health and the Environment (RIVM) deployed a decentralised multifaceted implementation strategy. This strategy consisted of a protocol specifying minimum requirements for an AS system, supported by a user manual, education module, individual guidance for hospitals and user-group meetings. This study describes an effect evaluation and process evaluation of the implementation strategy for AS in five frontrunner hospitals.

METHODS

To evaluate the effect of the implementation strategy, the achieved phase of implementation was determined in each frontrunner hospital at the end of the study period. The process evaluation consisted of (1) an evaluation of the feasibility of strategy elements, (2) an evaluation of barriers and facilitators for implementation and (3) an evaluation of the workload for implementation. Interviews were performed as a basis for a subsequent survey quantifying the results regarding the feasibility as well as barriers and facilitators. Workload was self-monitored per profession. Qualitative data were analysed using a framework analysis, whereas quantitative data were analysed descriptively.

RESULTS

One hospital finished the complete implementation process in 240 person-hours. Overall, the elements of the implementation strategy were often used, positively received and overall, the strategy was rated effective and feasible. During the implementation process, participants perceived the relative advantage of AS and had sufficient knowledge about AS. However, barriers regarding complexity of AS data extraction, data-infrastructure, and validation, lack of capacity and motivation at the IT department, and difficulties with assigning roles and responsibilities were experienced.

CONCLUSION

A decentralised multifaceted implementation strategy is suitable for the implementation of AS in hospitals. Effective local project management, including clear project leadership and ownership, obtaining commitment of higher management levels, active involvement of stakeholders, and appropriate allocation of roles and responsibilities is important for successful implementation and should be facilitated by the implementation strategy. Sufficient knowledge about AS, its requirements and the implementation process should be available among stakeholders by e.g. an education module. Furthermore, exchange of knowledge and experiences between hospitals should be encouraged in user-group meetings.

Surveillance of catheter-associated bloodstream infections: development and validation of a fully automated algorithm

BACKGROUND

Most surveillance systems for catheter-related bloodstream infections (CRBSI) and central line-associated bloodstream infections (CLABSI) are based on manual chart review. Our objective was to validate a fully automated algorithm for CRBSI and CLABSI surveillance in intensive care units (ICU).

METHODS

We developed a fully automated algorithm to detect CRBSI, CLABSI and ICU-onset bloodstream infections (ICU-BSI) in patients admitted to the ICU of a tertiary care hospital in Switzerland. The parameters included in the algorithm were based on a recently performed systematic review. Structured data on demographics, administrative data, central vascular catheter and microbiological results (blood cultures and other clinical cultures) obtained from the hospital's data warehouse were processed by the algorithm. Validation for CRBSI was performed by comparing results with prospective manual BSI surveillance data over a 6-year period. CLABSI were retrospectively assessed over a 2-year period.

RESULTS

From January 2016 to December 2021, 854 positive blood cultures were identified in 346 ICU patients. The median age was 61.7 years [IQR 50-70]; 205 (24%) positive samples were collected from female patients. The algorithm detected 5 CRBSI, 109 CLABSI and 280 ICU-BSI. The overall CRBSI and CLABSI incidence rates determined by automated surveillance for the period 2016 to 2021 were 0.18/1000 catheter-days (95% CI 0.06-0.41) and 3.86/1000 catheter days (95% CI: 3.17-4.65). The sensitivity, specificity, positive predictive and negative predictive values of the algorithm for CRBSI, were 83% (95% CI 43.7-96.9), 100% (95% CI 99.5-100), 100% (95% CI 56.5-100), and 99.9% (95% CI 99.2-100), respectively. One CRBSI was misclassified as an ICU-BSI by the algorithm because the same bacterium was identified in the blood culture and in a lower respiratory tract specimen. Manual review of CLABSI from January 2020 to December 2021 (n = 51) did not identify any errors in the algorithm.

CONCLUSIONS

A fully automated algorithm for CRBSI and CLABSI detection in critically-ill patients using only structured data provided valid results. The next step will be to assess the feasibility and external validity of implementing it in several hospitals with different electronic health record systems.

Development of machine learning models for the surveillance of colon surgical site infections

BACKGROUND

Conventional surgical site infection (SSI) surveillance is labour-intensive. We aimed to develop machine learning (ML) models for the surveillance of SSIs for colon surgery and to assess whether the ML could improve surveillance process efficiency.

METHODS

This study included cases who underwent colon surgery at a tertiary center between 2013 and 2014. Logistic regression and four ML algorithms including random forest (RF), gradient boosting (GB), and neural networks (NNs) with or without recursive feature elimination (RFE) were first trained on the entire cohort, and then re-trained on cases selected based on a previous rule-based algorithm. We assessed model performance based on the area under the curve (AUC), sensitivity, and positive predictive value (PPV). The estimated proportion of reduction in workload for chart review based on the ML models was evaluated and compared with the conventional method.

RESULTS

At a sensitivity of 95%, the NN with RFE using 29 variables had the best performance with an AUC of 0.963 and PPV of 21.1%. When combining both the rule-based algorithm and ML algorithms, the NN with RFE using 19 variables had a higher PPV (28.9%) than with the ML algorithm alone, which could decrease the number of cases requiring chart review by 83.9% compared with the conventional method.

CONCLUSION

We demonstrated that ML can improve the efficiency of SSI surveillance for colon surgery by decreasing the burden of chart review while providing high sensitivity. In particular, the hybrid approach of ML with a rule-based algorithm showed the best performance in terms of PPV.

Establishing semi-automated infection surveillance in obstetrics and gynaecology

BACKGROUND

Surveillance is an acknowledged method to decrease nosocomial infections, such as surgical site infections (SSIs). Electronic healthcare records create the opportunity for automated surveillance. While approaches for different types of surgeries and indicators already exist, there are very few for obstetrics and gynaecology.

AIM

To analyse the sensitivity and workload reduction of semi-automated surveillance in obstetrics and gynaecology.

METHODS

In this retrospective, single-centre study at a 1438-bed tertiary care hospital in Germany, semi-automated SSI surveillance using the indicators 'antibiotic prescription', 'microbiological data' and 'administrative data' (diagnosis codes, readmission, post-hospitalization care) was compared with manual analysis and categorization of all patient files. Breast surgeries (BSs) conducted in 2018 and caesarean sections (CSs) that met the inclusion criteria between May 2013 and December 2019 were included. Indicators were analysed for sensitivity, number of analysed procedures needed to identify one case, and potential workload reduction in detecting SSIs in comparison with the control group.

FINDINGS

The reference standard showed nine SSIs in 416 BSs (2.2%). Sensitivities for the indicators 'antibiotic prescription', 'diagnosis code', 'microbiological sample taken', and the combination 'diagnosis code or microbiological sample' were 100%, 88.9%, 66.7% and 100%, respectively. The reference standard showed 54 SSIs in 3438 CSs (1.6%). Sensitivities for the indicators 'collection of microbiological samples', 'diagnosis codes', 'readmission/post-hospitalization care', and the combination of all indicators were 38.9%, 27.8%, 85.2% and 94.4%, respectively.

CONCLUSIONS

Semi-automated surveillance systems may reduce workload by maintaining high sensitivity depending on the type of surgery, local circumstances and thorough digitalization.

Systematic scoping review of automated systems for the surveillance of healthcare-associated bloodstream infections related to intravascular catheters

INTRODUCTION

Intravascular catheters are crucial devices in medical practice that increase the risk of healthcare-associated infections (HAIs), and related health-economic adverse outcomes. This scoping review aims to provide a comprehensive overview of published automated algorithms for surveillance of catheter-related bloodstream infections (CRBSI) and central line-associated bloodstream infections (CLABSI).

METHODS

We performed a scoping review based on a systematic search of the literature in PubMed and EMBASE from 1 January 2000 to 31 December 2021. Studies were included if they evaluated predictive performance of automated surveillance algorithms for CLABSI/CRBSI detection and used manually collected surveillance data as reference. We assessed the design of the automated systems, including the definitions used to develop algorithms (CLABSI versus CRBSI), the datasets and denominators used, and the algorithms evaluated in each of the studies.

RESULTS

We screened 586 studies based on title and abstract, and 99 were assessed based on full text. Nine studies were included in the scoping review. Most studies were monocentric (n = 5), and they identified CLABSI (n = 7) as an outcome. The majority of the studies used administrative and microbiological data (n = 9) and five studies included the presence of a vascular central line in their automated system. Six studies explained the denominator they selected, five of which chose central line-days. The most common rules and steps used in the algorithms were categorized as hospital-acquired rules, infection rules (infection versus contamination), deduplication, episode grouping, secondary BSI rules (secondary versus primary BSI), and catheter-associated rules.

CONCLUSION

The automated surveillance systems that we identified were heterogeneous in terms of definitions, datasets and denominators used, with a combination of rules in each algorithm. Further guidelines and studies are needed to develop and implement algorithms to detect CLABSI/CRBSI, with standardized definitions, appropriate data sources and suitable denominators.

How a laboratory-based antimicrobial resistance (AMR) regional surveillance system can address large-scale and local AMR epidemiology: the MICRO-BIO experience

Antimicrobial resistance is a significant threat to public health, with Italy experiencing substantial challenges in terms of AMR rate, surveillance system and activities to combat AMR. In response, the MICRO-BIO project was initiated as part of the National Plan to Combat Antibiotic Resistance by Region Lombardy health department. It was launched in 2018 with the aim of creating a surveillance tool by integrating data on bacterial isolates from microbiology laboratories. The participating laboratories were directly involved in reviewing and addressing discrepancies in the transmission data quality assessment. Despite the disruptions caused by COVID-19, 30 out of 33 laboratories in the Lombardy Region were successfully integrated by October 2023, with 1,201,000 microbiological data collected in the first nine months of 2023. In 2022 the analysis yielded 15,037 blood culture results from 20 labs passing validation. Data regarding the antimicrobial resistance profile of high-priority pathogens was analyzed at regional and single-hospital levels. The MICRO-BIO project represents a significant step toward strengthening AMR surveillance in a highly populated region. As a multi-disciplinary tool encompassing the fields of public health and IT (information technology), this tool has the potential to inform regional and local AMR epidemiology.

Comorbidities directly extracted from the hospital database for adjusting SSI risk in the new national semiautomated surveillance system in France: The SPICMI network

OBJECTIVE

To evaluate the performance of a comorbidity-based risk-adjustment model for surgical-site infection (SSI) reporting and benchmarking using a panel of variables extracted from the hospital discharge database (HDD), including comorbidities, compared to other models that use variables from different data sources.

METHODS

The French national surveillance program for SSI (SPICMI) has collected data from voluntary hospitals in the first 6 months of 2020 and 2021, for 16 selected surgery procedures, using a semiautomated algorithm for detection. Four risk-adjustment models were selected with logistic regression analysis, combining the different patterns of variables: National Nosocomial Infections Surveillance System (NNIS) risk-index components, individual operative data, and 6 individual comorbidities according to International Classification of Disease, Tenth Revision (ICD-10) diagnosis: obesity, diabetes, malnutrition, hypertension, cancer, or immunosuppression. Areas under the curve (AUCs) were calculated and compared.

RESULTS

Overall, 294 SSI were detected among 11,975 procedures included. All 6 comorbidities were related to SSI in the univariate analysis. The AUC of the selected model including comorbidities (0.675; 95% confidence interval [CI], 0.642-0.707), was significantly higher than the AUC of the model without comorbidities (0.641; 95% CI, 0.609-0.672; P = .016) or the AUC using the NNIS-index components (0.598; 95% CI, 0.564-0.630; P < .001). The HDD-based model AUC (0.659; 95% CI, 0.625-0.692) did not differ significantly from the selected model without comorbidities (P = .23).

CONCLUSION

Including HDD-based comorbidities as patient case-mix variables instead of NNIS risk index factors could be an effective approach for risk-adjustment of automated SSI surveillance more widely accessible to hospitals.

Predictive performance of automated surveillance algorithms for intravascular catheter bloodstream infections: a systematic review and meta-analysis

BACKGROUND

Intravascular catheter infections are associated with adverse clinical outcomes. However, a significant proportion of these infections are preventable. Evaluations of the performance of automated surveillance systems for adequate monitoring of central-line associated bloodstream infection (CLABSI) or catheter-related bloodstream infection (CRBSI) are limited.

OBJECTIVES

We evaluated the predictive performance of automated algorithms for CLABSI/CRBSI detection, and investigated which parameters included in automated algorithms provide the greatest accuracy for CLABSI/CRBSI detection.

METHODS

We performed a meta-analysis based on a systematic search of published studies in PubMed and EMBASE from 1 January 2000 to 31 December 2021. We included studies that evaluated predictive performance of automated surveillance algorithms for CLABSI/CRBSI detection and used manually collected surveillance data as reference. We estimated the pooled sensitivity and specificity of algorithms for accuracy and performed a univariable meta-regression of the different parameters used across algorithms.

RESULTS

The search identified five full text studies and 32 different algorithms or study populations were included in the meta-analysis. All studies analysed central venous catheters and identified CLABSI or CRBSI as an outcome. Pooled sensitivity and specificity of automated surveillance algorithm were 0.88 [95%CI 0.84-0.91] and 0.86 [95%CI 0.79-0.92] with significant heterogeneity (I2 = 91.9, p < 0.001 and I2 = 99.2, p < 0.001, respectively). In meta-regression, algorithms that include results of microbiological cultures from specific specimens (respiratory, urine and wound) to exclude non-CRBSI had higher specificity estimates (0.92, 95%CI 0.88-0.96) than algorithms that include results of microbiological cultures from any other body sites (0.88, 95% CI 0.81-0.95). The addition of clinical signs as a predictor did not improve performance of these algorithms with similar specificity estimates (0.92, 95%CI 0.88-0.96).

CONCLUSIONS

Performance of automated algorithms for detection of intravascular catheter infections in comparison to manual surveillance seems encouraging. The development of automated algorithms should consider the inclusion of results of microbiological cultures from specific specimens to exclude non-CRBSI, while the inclusion of clinical data may not have an added-value. Trail Registration Prospectively registered with International prospective register of systematic reviews (PROSPERO ID CRD42022299641; January 21, 2022). https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42022299641.

Factors Influencing Clinicians' Use of Hospital Information Systems for Infection Prevention and Control: Cross-Sectional Study Based on the Extended DeLone and McLean Model

BACKGROUND

Healthcare-associated infections have become a serious public health problem. Various types of information systems have begun to be applied in hospital infection prevention and control (IPC) practice. Clinicians are the key users of these systems, but few studies have assessed the use of infection prevention and control information systems (IPCISs) from their perspective.

OBJECTIVE

This study aimed to (1) apply the extended DeLone and McLean Information Systems Success model (D&M model) that incorporates IPC culture to examine how technical factors like information quality, system quality, and service quality, as well as organizational culture factors affect clinicians' use intention, satisfaction, and perceived net benefits, and (2) identify which factors are the most important for clinicians' use intention.

METHODS

A total of 12,317 clinicians from secondary and tertiary hospitals were surveyed online. Data were analyzed using partial least squares-structural equation modeling and the importance-performance matrix analysis.

RESULTS

Among the technical factors, system quality (β=.089-.252; P<.001), information quality (β=.294-.102; P<.001), and service quality (β=.126-.411; P<.001) were significantly related to user satisfaction (R2=0.833), use intention (R2=0.821), and perceived net benefits (communication benefits [R2=0.676], decision-making benefits [R2=0.624], and organizational benefits [R2=0.656]). IPC culture had an effect on use intention (β=.059; P<.001), and it also indirectly affected perceived net benefits (β=.461-.474; P<.001). In the importance-performance matrix analysis, the attributes of service quality (providing user training) and information quality (readability) were present in the fourth quadrant, indicating their high importance and low performance.

CONCLUSIONS

This study provides valuable insights into IPCIS usage among clinicians from the perspectives of technology and organization culture factors. It found that technical factors (system quality, information quality, and service quality) and hospital IPC culture have an impact on the successful use of IPCISs after evaluating the application of IPCISs based on the extended D&M model. Furthermore, service quality and information quality showed higher importance and lower performance for use intention. These findings provide empirical evidence and specific practical directions for further improving the construction of IPCISs.

Surgical site infection surveillance in German hospitals: a national survey to determine the status quo of digitalization

BACKGROUND

Surveillance of surgical site infections (SSI) relies on access to data from various sources. Insights into the practices of German hospitals conducting SSI surveillance and their information technology (IT) infrastructures are scarce. The aim of this study was to evaluate current SSI surveillance practices in German hospitals with a focus on employed IT infrastructures.

METHODS

German surgical departments actively participating in the national SSI surveillance module "OP-KISS" were invited in August 2020 to participate in a questionnaire-based online survey. Depending on whether departments entered all data manually or used an existing feature to import denominator data into the national surveillance database, departments were separated into different groups. Selected survey questions differed between groups.

RESULTS

Of 1,346 invited departments, 821 participated in the survey (response rate: 61%). Local IT deficits (n = 236), incompatibility of import specifications and hospital information system (n = 153) and lack of technical expertise (n = 145) were cited as the most frequent reasons for not using the denominator data import feature. Conversely, reduction of workload (n = 160) was named as the main motivation to import data. Questions on data availability and accessibility in the electronic hospital information system (HIS) and options to export data from the HIS for the purpose of surveillance, yielded diverse results. Departments utilizing the import feature tended to be from larger hospitals with a higher level of care.

CONCLUSIONS

The degree to which digital solutions were employed for SSI surveillance differed considerably between surgical departments in Germany. Improving availability and accessibility of information in HIS and meeting interoperability standards will be prerequisites for increasing the amount of data exported directly from HIS to national databases and laying the foundation for automated SSI surveillance on a broad scale.

Clusters of healthcare-associated SARS-CoV-2 infections in Norwegian hospitals detected by a fully automatic register-based surveillance system

BACKGROUND

Notifications to the Norwegian Institute of Public Health of outbreaks in Norwegian healthcare institutions are mandatory by law, but underreporting is suspected due to failure to identify clusters, or because of human or system-based factors. This study aimed to establish and describe a fully automatic, register-based surveillance system to identify clusters of healthcare-associated infections (HAIs) of SARS-CoV-2 in hospitals and compare these with outbreaks notified through the mandated outbreak system Vesuv.

METHODS

We used linked data from the emergency preparedness register Beredt C19, based on the Norwegian Patient Registry and the Norwegian Surveillance System for Communicable Diseases. We tested two different algorithms for HAI clusters, described their size and compared them to outbreaks notified through Vesuv.

RESULTS

5033 patients were registered with an indeterminate, probable, or definite HAI. Depending on the algorithm, our system detected 44 or 36 of the 56 officially notified outbreaks. Both algorithms detected more clusters then officially reported (301 and 206, respectively).

CONCLUSIONS

It was possible to use existing data sources to establish a fully automatic surveillance system identifying clusters of SARS-CoV-2 s. Automatic surveillance can improve preparedness through earlier identification of clusters of HAIs, and by lowering the workloads of infection control specialists in hospitals.

The future of automated infection detection: Innovation to transform practice (Part III/III)

Current methods of emergency-room-based syndromic surveillance were insufficient to detect early community spread of severe acute respiratory coronavirus virus 2 (SARS-CoV-2) in the United States, which slowed the infection prevention and control response to the novel pathogen. Emerging technologies and automated infection surveillance have the potential to improve upon current practice standards and to revolutionize the practice of infection detection, prevention and control both inside and outside of healthcare settings. Genomics, natural language processing, and machine learning can be leveraged to improve identification of transmission events and aid and evaluate outbreak response. In the near future, automated infection detection strategies can be used to advance a true "Learning Healthcare System" that will support near-real-time quality improvement efforts and advance the scientific basis for the practice of infection control.

Validating administrative data to identify complex surgical site infections following cardiac implantable electronic device implantation: a comparison of traditional methods and machine learning

Background

Cardiac implantable electronic device (CIED) surgical site infections (SSIs) have been outpacing the increases in implantation of these devices. While traditional surveillance of these SSIs by infection prevention and control would likely be the most accurate, this is not practical in many centers where resources are constrained. Therefore, we explored the validity of administrative data at identifying these SSIs.

Methods

We used a cohort of all patients with CIED implantation in Calgary, Alberta where traditional surveillance was done for infections from Jan 1, 2013 to December 31, 2019. We used this infection subgroup as our “gold standard” and then utilized various combinations of administrative data to determine which best optimized the sensitivity and specificity at identifying infection. We evaluated six approaches to identifying CIED infection using administrative data, which included four algorithms using International Classification of Diseases codes and/or Canadian Classification of Health Intervention codes, and two machine learning models. A secondary objective of our study was to assess if machine learning techniques with training of logistic regression models would outperform our pre-selected codes.

Results

We determined that all of the pre-selected algorithms performed well at identifying CIED infections but the machine learning model was able to produce the optimal method of identification with an area under the receiver operating characteristic curve (AUC) of 96.8%. The best performing pre-selected algorithm yielded an AUC of 94.6%.

Conclusions

Our findings suggest that administrative data can be used to effectively identify CIED infections. While machine learning performed the most optimally, in centers with limited analytic capabilities a simpler algorithm of pre-selected codes also has excellent yield. This can be valuable for centers without traditional surveillance to follow trends in SSIs over time and identify when rates of infection are increasing. This can lead to enhanced interventions for prevention of SSIs.

Semiautomated surveillance of deep surgical site infections after colorectal surgeries: A multicenter external validation of two surveillance algorithms

Objective:

Automated surveillance methods increasingly replace or support conventional (manual) surveillance; the latter is labor intensive and vulnerable to subjective interpretation. We sought to validate 2 previously developed semiautomated surveillance algorithms to identify deep surgical site infections (SSIs) in patients undergoing colorectal surgeries in Dutch hospitals.

Design:

Multicenter retrospective cohort study.

Methods:

From 4 hospitals, we selected colorectal surgery patients between 2018 and 2019 based on procedure codes, and we extracted routine care data from electronic health records. Per hospital, a classification model and a regression model were applied independently to classify patients into low- or high probability of having developed deep SSI. High-probability patients need manual SSI confirmation; low-probability records are classified as no deep SSI. Sensitivity, positive predictive value (PPV), and workload reduction were calculated compared to conventional surveillance.

Results:

In total, 672 colorectal surgery patients were included, of whom 28 (4.1%) developed deep SSI. Both surveillance models achieved good performance. After adaptation to clinical practice, the classification model had 100% sensitivity and PPV ranged from 11.1% to 45.8% between hospitals. The regression model had 100% sensitivity and 9.0%–14.9% PPV. With both models, <25% of records needed review to confirm SSI. The regression model requires more complex data management skills, partly due to incomplete data.

Conclusions:

In this independent external validation, both surveillance models performed well. The classification model is preferred above the regression model because of source-data availability and less complex data-management requirements. The next step is implementation in infection prevention practices and workflow processes.