Automatic detection of patients with nosocomial infection by a computer-based surveillance system: a validation study in a general hospital

Establishing semi-automated infection surveillance in obstetrics and gynaecology

BACKGROUND

Surveillance is an acknowledged method to decrease nosocomial infections, such as surgical site infections (SSIs). Electronic healthcare records create the opportunity for automated surveillance. While approaches for different types of surgeries and indicators already exist, there are very few for obstetrics and gynaecology.

AIM

To analyse the sensitivity and workload reduction of semi-automated surveillance in obstetrics and gynaecology.

METHODS

In this retrospective, single-centre study at a 1438-bed tertiary care hospital in Germany, semi-automated SSI surveillance using the indicators 'antibiotic prescription', 'microbiological data' and 'administrative data' (diagnosis codes, readmission, post-hospitalization care) was compared with manual analysis and categorization of all patient files. Breast surgeries (BSs) conducted in 2018 and caesarean sections (CSs) that met the inclusion criteria between May 2013 and December 2019 were included. Indicators were analysed for sensitivity, number of analysed procedures needed to identify one case, and potential workload reduction in detecting SSIs in comparison with the control group.

FINDINGS

The reference standard showed nine SSIs in 416 BSs (2.2%). Sensitivities for the indicators 'antibiotic prescription', 'diagnosis code', 'microbiological sample taken', and the combination 'diagnosis code or microbiological sample' were 100%, 88.9%, 66.7% and 100%, respectively. The reference standard showed 54 SSIs in 3438 CSs (1.6%). Sensitivities for the indicators 'collection of microbiological samples', 'diagnosis codes', 'readmission/post-hospitalization care', and the combination of all indicators were 38.9%, 27.8%, 85.2% and 94.4%, respectively.

CONCLUSIONS

Semi-automated surveillance systems may reduce workload by maintaining high sensitivity depending on the type of surgery, local circumstances and thorough digitalization.

Healthcare-associated infections and conditions in the era of digital measurement

As the third edition of the Compendium of Strategies to Prevent Healthcare-Associated Infections in Acute Care Hospitals is released with the latest recommendations for the prevention and management of healthcare-associated infections (HAIs), a new approach to reporting HAIs is just beginning to unfold. This next generation of HAI reporting will be fully electronic and based largely on existing data in electronic health record (EHR) systems and other electronic data sources. It will be a significant change in how hospitals report HAIs and how the Centers for Disease Control and Prevention (CDC) and other agencies receive this information. This paper outlines what that future electronic reporting system will look like and how it will impact HAI reporting.

Electronically assisted surveillance systems of healthcare-associated infections: a systematic review

Background

Surveillance of healthcare-associated infections (HAI) is the basis of each infection control programme and, in case of acute care hospitals, should ideally include all hospital wards, medical specialties as well as all types of HAI. Traditional surveillance is labour intensive and electronically assisted surveillance systems (EASS) hold the promise to increase efficiency.

Objectives

To give insight in the performance characteristics of different approaches to EASS and the quality of the studies designed to evaluate them.

Methods

In this systematic review, online databases were searched and studies that compared an EASS with a traditional surveillance method were included. Two different indicators were extracted from each study, one regarding the quality of design (including reporting efficiency) and one based on the performance (e.g. specificity and sensitivity) of the EASS presented.

Results

A total of 78 studies were included. The majority of EASS (n = 72) consisted of an algorithm-based selection step followed by confirmatory assessment. The algorithms used different sets of variables. Only a minority (n = 7) of EASS were hospital-wide and designed to detect all types of HAI. Sensitivity of EASS was generally high (> 0.8), but specificity varied (0.37–1). Less than 20% (n = 14) of the studies presented data on the efficiency gains achieved.

Conclusions

Electronically assisted surveillance of HAI has yet to reach a mature stage and to be used routinely in healthcare settings. We recommend that future studies on the development and implementation of EASS of HAI focus on thorough validation, reproducibility, standardised datasets and detailed information on efficiency.

Cloud Computing for Infectious Disease Surveillance and Control: Development and Evaluation of a Hospital Automated Laboratory Reporting System

BACKGROUND

Outbreaks of several serious infectious diseases have occurred in recent years. In response, to mitigate public health risks, countries worldwide have dedicated efforts to establish an information system for effective disease monitoring, risk assessment, and early warning management for international disease outbreaks. A cloud computing framework can effectively provide the required hardware resources and information access and exchange to conveniently connect information related to infectious diseases and develop a cross-system surveillance and control system for infectious diseases.

OBJECTIVE

The objective of our study was to develop a Hospital Automated Laboratory Reporting (HALR) system based on such a framework and evaluate its effectiveness.

METHODS

We collected data for 6 months and analyzed the cases reported within this period by the HALR and the Web-based Notifiable Disease Reporting (WebNDR) systems. Furthermore, system evaluation indicators were gathered, including those evaluating sensitivity and specificity.

RESULTS

The HALR system reported 15 pathogens and 5174 cases, and the WebNDR system reported 34 cases. In a comparison of the two systems, sensitivity was 100% and specificity varied according to the reported pathogens. In particular, the specificity for Streptococcus pneumoniae, Mycobacterium tuberculosis complex, and hepatitis C virus were 99.8%, 96.6%, and 97.4%, respectively. However, the specificity for influenza virus and hepatitis B virus were only 79.9% and 47.1%, respectively. After the reported data were integrated with patients' diagnostic results in their electronic medical records (EMRs), the specificity for influenza virus and hepatitis B virus increased to 89.2% and 99.1%, respectively.

CONCLUSIONS

The HALR system can provide early reporting of specified pathogens according to test results, allowing for early detection of outbreaks and providing trends in infectious disease data. The results of this study show that the sensitivity and specificity of early disease detection can be increased by integrating the reported data in the HALR system with the cases' clinical information (eg, diagnostic results) in EMRs, thereby enhancing the control and prevention of infectious diseases.

National Automated Surveillance of Hospital-Acquired Bacteremia in Denmark Using a Computer Algorithm

BACKGROUND In 2015, Denmark launched an automated surveillance system for hospital-acquired infections, the Hospital-Acquired Infections Database (HAIBA). OBJECTIVE To describe the algorithm used in HAIBA, to determine its concordance with point prevalence surveys (PPSs), and to present trends for hospital-acquired bacteremia SETTING Private and public hospitals in Denmark METHODS A hospital-acquired bacteremia case was defined as at least 1 positive blood culture with at least 1 pathogen (bacterium or fungus) taken between 48 hours after admission and 48 hours after discharge, using the Danish Microbiology Database and the Danish National Patient Registry. PPSs performed in 2012 and 2013 were used for comparison. RESULTS National trends showed an increase in HA bacteremia cases between 2010 and 2014. Incidence was higher for men than women (9.6 vs 5.4 per 10,000 risk days) and was highest for those aged 61-80 years (9.5 per 10,000 risk days). The median daily prevalence was 3.1% (range, 2.1%-4.7%). Regional incidence varied from 6.1 to 8.1 per 10,000 risk days. The microorganisms identified were typical for HA bacteremia. Comparison of HAIBA with PPS showed a sensitivity of 36% and a specificity of 99%. HAIBA was less sensitive for patients in hematology departments and intensive care units. Excluding these departments improved the sensitivity of HAIBA to 44%. CONCLUSIONS Although the estimated sensitivity of HAIBA compared with PPS is low, a PPS is not a gold standard. Given the many advantages of automated surveillance, HAIBA allows monitoring of HA bacteremia across the healthcare system, supports prioritizing preventive measures, and holds promise for evaluating interventions. Infect Control Hosp Epidemiol 2017;38:559-566.

Automated surveillance system for hospital-acquired urinary tract infections in Denmark

BACKGROUND

The Danish Hospital-Acquired Infections Database (HAIBA) is an automated surveillance system using hospital administrative, microbiological, and antibiotic medication data.

AIM

To define and evaluate the case definition for hospital-acquired urinary tract infection (HA-UTI) and to describe surveillance data from 2010 to 2014.

METHODS

The HA-UTI algorithm defined a laboratory-diagnosed UTI as a urine culture positive for no more than two micro-organisms with at least one at ≥10(4)cfu/mL, and a probable UTI as a negative urine culture and a relevant diagnosis code or antibiotic treatment. UTI was considered hospital-acquired if a urine sample was collected ≥48h after admission and <48h post discharge. Incidence of HA-UTI was calculated per 10,000 risk-days. For validation, prevalence was calculated for each day and compared to point prevalence survey (PPS) data.

FINDINGS

HAIBA detected a national incidence rate of 42.2 laboratory-diagnosed HA-UTI per 10,000 risk-days with an increasing trend. Compared to PPS the laboratory-diagnosed HA-UTI algorithm had a sensitivity of 50.0% (26/52) and a specificity of 94.2% (1842/1955). There were several reasons for discrepancies between HAIBA and PPS, including laboratory results being unavailable at the time of the survey, the results considered clinically irrelevant by the surveyor due to an indwelling urinary catheter or lack of clinical signs of infection, and UTIs being considered HA-UTI in PPS even though the first sample was taken within 48h of admission.

CONCLUSION

The HAIBA algorithm was found to give valid and valuable information and has, among others, the advantages of covering the whole population and allowing continuous standardized monitoring of HA-UTI.

A Web-Based, Hospital-Wide Health Care-Associated Bloodstream Infection Surveillance and Classification System: Development and Evaluation

Background

Surveillance of health care-associated infections is an essential component of infection prevention programs, but conventional systems are labor intensive and performance dependent.

Objective

To develop an automatic surveillance and classification system for health care-associated bloodstream infection (HABSI), and to evaluate its performance by comparing it with a conventional infection control personnel (ICP)-based surveillance system.

Methods

We developed a Web-based system that was integrated into the medical information system of a 2200-bed teaching hospital in Taiwan. The system automatically detects and classifies HABSIs.

Results

In this study, the number of computer-detected HABSIs correlated closely with the number of HABSIs detected by ICP by department (n=20; r=.999 P<.001) and by time (n=14; r=.941; P<.001). Compared with reference standards, this system performed excellently with regard to sensitivity (98.16%), specificity (99.96%), positive predictive value (95.81%), and negative predictive value (99.98%). The system enabled decreasing the delay in confirmation of HABSI cases, on average, by 29 days.

Conclusions

This system provides reliable and objective HABSI data for quality indicators, improving the delay caused by a conventional surveillance system.

Accuracy of administrative data for surveillance of healthcare-associated infections: a systematic review

OBJECTIVE

Measuring the incidence of healthcare-associated infections (HAI) is of increasing importance in current healthcare delivery systems. Administrative data algorithms, including (combinations of) diagnosis codes, are commonly used to determine the occurrence of HAI, either to support within-hospital surveillance programmes or as free-standing quality indicators. We conducted a systematic review evaluating the diagnostic accuracy of administrative data for the detection of HAI.

METHODS

Systematic search of Medline, Embase, CINAHL and Cochrane for relevant studies (1995-2013). Methodological quality assessment was performed using QUADAS-2 criteria; diagnostic accuracy estimates were stratified by HAI type and key study characteristics.

RESULTS

57 studies were included, the majority aiming to detect surgical site or bloodstream infections. Study designs were very diverse regarding the specification of their administrative data algorithm (code selections, follow-up) and definitions of HAI presence. One-third of studies had important methodological limitations including differential or incomplete HAI ascertainment or lack of blinding of assessors. Observed sensitivity and positive predictive values of administrative data algorithms for HAI detection were very heterogeneous and generally modest at best, both for within-hospital algorithms and for formal quality indicators; accuracy was particularly poor for the identification of device-associated HAI such as central line associated bloodstream infections. The large heterogeneity in study designs across the included studies precluded formal calculation of summary diagnostic accuracy estimates in most instances.

CONCLUSIONS

Administrative data had limited and highly variable accuracy for the detection of HAI, and their judicious use for internal surveillance efforts and external quality assessment is recommended. If hospitals and policymakers choose to rely on administrative data for HAI surveillance, continued improvements to existing algorithms and their robust validation are imperative.

Development of a Novel Electronic Surveillance System for Monitoring of Bloodstream Infections

Background.

Electronic surveillance systems (ESSs) that utilize existing information in databases are more efficient than conventional infection surveillance methods.

Objective.

To develop an ESS for monitoring bloodstream infections (BSIs) and assess whether data obtained from the ESS were in agreement with data obtained by traditional manual medical-record review.

Methods.

An ESS was developed by linking data from regional laboratory and hospital administrative databases. Definitions for excluding BSI episodes representing contamination and duplicate episodes were developed and applied. Infections were classified as nosocomial infections, healthcare-associated community-onset infections, or community-acquired infections. For a random sample of episodes, data in the ESS were compared with data obtained by independent medical chart review.

Results.

From the records of the 306 patients whose infections were selected for comparative review, the ESS identified 323 episodes of BSI, of which 107 (33%) were classified as healthcare-associated community-onset infections, 108 (33%) were classified as community-acquired infections, 107 (33%) were classified as nosocomial infections, and 1 (0.3%) could not be classified. In comparison, 310 episodes were identified by use of medical chart review, of which 116 (37%) were classified as healthcare-associated community-onset infections, 95 (31%) as community-acquired infections, and 99 (32%) as nosocomial infections. For 302 episodes of BSI, there was concordance between the findings of the ESS and those of traditional manual chart review. Of the additional 21 discordant episodes that were identified by use of the ESS, 17 (81%) were classified as representing isolation of skin contaminants, by use of chart review. Of the additional 8 discordant episodes further identified by use of chart review, most were classified as repeat or polymicrobial episodes of disease. There was an overall 85% agreement between the findings of the ESS and those of chart review (K = 0.78; standard error, K = 0.04) for classification according to location of acquisition.

Conclusion.

Our novel ESS allows episodes of BSI to be identified and classified with a high degree of accuracy. This system requires validation in other cohorts and settings.

Improving the work efficiency of healthcare-associated infection surveillance using electronic medical records

In this study, we developed an integrated hospital-associated urinary tract infection (HAUTI) surveillance information system (called iHAUTISIS) based on existing electronic medical records (EMR) systems for improving the work efficiency of infection control professionals (ICPs) in a 730-bed, tertiary-care teaching hospital in Taiwan. The iHAUTISIS can automatically collect data relevant to HAUTI surveillance from the different EMR systems, and provides a visualization dashboard that helps ICPs make better surveillance plans and facilitates their surveillance work. In order to measure the system performance, we also created a generic model for comparing the ICPs' work efficiency when using existing electronic culture-based surveillance information system (eCBSIS) and iHAUTISIS, respectively. This model can demonstrate a patient's state (unsuspected, suspected, and confirmed) and corresponding time spent on surveillance tasks performed by ICPs for the patient in that state. The study results showed that the iHAUTISIS performed better than the eCBSIS in terms of ICPs' time cost. It reduced the time by 73.27 s, when using iHAUTISIS (114.26 s) and eCBSIS (187.53 s), for each patient on average. With increased adoption of EMR systems, the development of the integrated HAI surveillance information systems would be more and more cost-effective. Moreover, the iHAUTISIS adopted web-based technology that enables ICPs to online access patient's surveillance information using laptops or mobile devices. Therefore, our system can further facilitate the HAI surveillance and reduce ICPs' surveillance workloads.

Identifying colon and open reduction of fracture surgical site infections using a partially automated electronic algorithm

BACKGROUND

Algorithms leveraging electronic data may reduce manual review burden for surgical site infection (SSI) surveillance with little to no reduction in sensitivity. We developed an algorithm to identify colon and open reduction of fracture (FX) SSIs to reduce manual chart review.

METHODS

A retrospective cohort of colon and FX procedures and associated SSIs was constructed. Potential SSIs were identified by positive microbiologic cultures or administrative data for diagnosis or treatment of wound infection. Sensitivity and specificity of the algorithm were assessed. The number of charts needing review to identify 1 SSI, and the potential time-savings from the algorithm, were calculated.

RESULTS

Four hundred seventy-three colon (SSI rate = 7%) and 1081 FX (SSI rate = 3%) procedures were identified. The algorithm was 91% and 97% sensitive and 76% and 93% specific for colon and FX procedures, respectively. Overall, chart review would have been reduced by 24.3 hours per 100 procedures, decreasing the number of charts to review to identify 1 SSI from 23.9 for manual review to 3.9 with the algorithm.

CONCLUSIONS

The algorithm identified SSIs with excellent sensitivity and specificity, resulting in substantial reductions in manual chart review. This algorithm could be tailored and applied to other hospitals.

An automated algorithm to preselect patients to be assessed individually in point prevalence surveys for hospital-acquired infections in surgery

In this pilot study, we evaluate an algorithm that uses predictive clinical and laboratory parameters to differentiate between patients with hospital-acquired infection (HAI) and patients without HAI. Seventy-four percent of the studied population of surgical patients could be reliably (negative predictive value of 98%) excluded from detailed assessment by the infection control practitioner.

Evaluation of an algorithm for electronic surveillance of hospital-acquired infections yielding serial weekly point prevalence scores

Surveillance of hospital-acquired infections can be approximated by repeated surveys that are performed in a standardized, cost-effective manner. We developed an integrated software system for serial electronic hospital-wide point prevalence surveys using algorithms that proved highly sensitive and specific over a 5-year period in a large university medical center.

Validity analysis of a unique infection surveillance system in the intensive care unit by analysis of a data warehouse built through a workflow-integrated software application

BACKGROUND

An electronic decision support programme was developed within the intensive care unit (ICU) that provides an overview of all infection-related patient data, and allows ICU physicians to add clinical information during patient rounds, resulting in prospective compilation of a database.

AIM

To assess the validity of computer-assisted surveillance (CAS) of ICU-acquired infection performed by analysis of this database.

METHODS

CAS was compared with prospective paper-based surveillance (PBS) for ICU-acquired respiratory tract infection (RTI), bloodstream infection (BSI) and urinary tract infection (UTI) over four months at a 36-bed medical and surgical ICU. An independent panel reviewed the data in the case of discrepancy between CAS and PBS.

FINDINGS

PBS identified 89 ICU-acquired infections (13 BSI, 18 UTI, 58 RTI) and CAS identified 90 ICU-acquired infections (14 BSI, 17 UTI, 59 RTI) in 876 ICU admissions. There was agreement between CAS and PBS on 13 BSI (100 %), 14 UTI (77.8 %) and 42 RTI (72.4 %). Overall, there was agreement on 69 infections (77.5%), resulting in a kappa score of 0.74. Discrepancy between PBS and CAS was the result of capture error in 11 and 14 infections, respectively. Interobserver disagreement on probability (13 RTI) and focus (two RTI, one UTI) occurred for 16 episodes. The time required to collect information using CAS is less than 30% of the time required when using PBS.

CONCLUSION

CAS for ICU-acquired infection by analysis of a database built through daily workflow is a feasible surveillance method and has good agreement with PBS. Discrepancy between CAS and PBS is largely due to interobserver variability.

Real-time automatic hospital-wide surveillance of nosocomial infections and outbreaks in a large Chinese tertiary hospital

Background

We aimed to develop a real-time nosocomial infection surveillance system (RT-NISS) to monitor all nosocomial infections (NIs) and outbreaks in a Chinese comprehensive hospital to better prevent and control NIs.

Methods

The screening algorithm used in RT-NISS included microbiological reports, antibiotic usage, serological and molecular testing, imaging reports, and fever history. The system could, in real-time, identify new NIs, record data, and produce time-series reports to align NI cases.

Results

Compared with a manual survey of NIs (the gold standard), the sensitivity and specificity of RT-NISS was 98.8% (84/85) and 93.0% (827/889), with time-saving efficiencies of about 200 times. RT-NISS obtained the highest hospital-wide monthly NI rate of 2.62%, while physician and medical record reviews reported rates of 1.52% and 2.35% respectively. It took about two hours for one infection control practitioner (ICP) to deal with 70 new suspicious NI cases; there were 3,500 inpatients each day in the study hospital. The system could also provide various updated data (i.e. the daily NI rate, surgical site infection (SSI) rate) for each ward, or the entire hospital. Within 3 years of implementing RT-NISS, the ICPs monitored and successfully controlled about 30 NI clusters and 4 outbreaks at the study hospital.

Conclusions

Just like the “ICPs’ eyes”, RT-NISS was an essential and efficient tool for the day-to-day monitoring of all NIs and outbreak within the hospital; a task that would not have been accomplished through manual process.

Data use and effectiveness in electronic surveillance of healthcare associated infections in the 21st century: a systematic review

OBJECTIVE

As more electronic health records have become available during the last decade, we aimed to uncover recent trends in use of electronically available patient data by electronic surveillance systems for healthcare associated infections (HAIs) and identify consequences for system effectiveness.

METHODS

A systematic review of published literature evaluating electronic HAI surveillance systems was performed. The PubMed service was used to retrieve publications between January 2001 and December 2011. Studies were included in the review if they accurately described what electronic data were used and if system effectiveness was evaluated using sensitivity, specificity, positive predictive value, or negative predictive value. Trends were identified by analyzing changes in the number and types of electronic data sources used.

RESULTS

26 publications comprising discussions on 27 electronic systems met the eligibility criteria. Trend analysis showed that systems use an increasing number of data sources which are either medico-administrative or clinical and laboratory-based data. Trends on the use of individual types of electronic data confirmed the paramount role of microbiology data in HAI detection, but also showed increased use of biochemistry and pharmacy data, and the limited adoption of clinical data and physician narratives. System effectiveness assessments indicate that the use of heterogeneous data sources results in higher system sensitivity at the expense of specificity.

CONCLUSIONS

Driven by the increased availability of electronic patient data, electronic HAI surveillance systems use more data, making systems more sensitive yet less specific, but also allow systems to be tailored to the needs of healthcare institutes' surveillance programs.

Diagnostic performance of electronic syndromic surveillance systems in acute care: a systematic review

CONTEXT

Healthcare Electronic Syndromic Surveillance (ESS) is the systematic collection, analysis and interpretation of ongoing clinical data with subsequent dissemination of results, which aid clinical decision-making.

OBJECTIVE

To evaluate, classify and analyze the diagnostic performance, strengths and limitations of existing acute care ESS systems.

DATA SOURCES

All available to us studies in Ovid MEDLINE, Ovid EMBASE, CINAHL and Scopus databases, from as early as January 1972 through the first week of September 2012.

STUDY SELECTION

Prospective and retrospective trials, examining the diagnostic performance of inpatient ESS and providing objective diagnostic data including sensitivity, specificity, positive and negative predictive values.

DATA EXTRACTION

Two independent reviewers extracted diagnostic performance data on ESS systems, including clinical area, number of decision points, sensitivity and specificity. Positive and negative likelihood ratios were calculated for each healthcare ESS system. A likelihood matrix summarizing the various ESS systems performance was created.

RESULTS

The described search strategy yielded 1639 articles. Of these, 1497 were excluded on abstract information. After full text review, abstraction and arbitration with a third reviewer, 33 studies met inclusion criteria, reporting 102,611 ESS decision points. The yielded I2 was high (98.8%), precluding meta-analysis. Performance was variable, with sensitivities ranging from 21% -100% and specificities ranging from 5%-100%.

CONCLUSIONS

There is significant heterogeneity in the diagnostic performance of the available ESS implements in acute care, stemming from the wide spectrum of different clinical entities and ESS systems. Based on the results, we introduce a conceptual framework using a likelihood ratio matrix for evaluation and meaningful application of future, frontline clinical decision support systems.

Advances in electronic surveillance for healthcare-associated infections in the 21st Century: a systematic review

BACKGROUND

Traditional methodologies for healthcare-associated infection (HCAI) surveillance can be resource intensive and time consuming. As a consequence, surveillance is often limited to specific organisms or conditions. Various electronic databases exist within the healthcare setting and may be utilized to perform HCAI surveillance.

AIM

To assess the utility of electronic surveillance systems for monitoring and detecting HCAI.

METHODS

A systematic review of published literature on surveillance of HCAI was performed. Databases were searched for studies published between January 2000 and December 2011. Search terms were divided into infection, surveillance and data management terms, and combined using Boolean operators. Studies were included for review if they demonstrated or proposed the use of electronic systems for HCAI surveillance.

FINDINGS

In total, 44 studies met the inclusion criteria. For the majority of studies, emphasis was on the linkage of electronic databases to provide automated methods for monitoring infections in specific clinical settings. Twenty-one studies assessed the performance of their method with traditional surveillance methodologies or a manual reference method. Where sensitivity and specificity were calculated, these varied depending on the organism or condition being surveyed and the data sources employed.

CONCLUSIONS

The implementation of electronic surveillance was found to be feasible in many settings, with several systems fully integrated into hospital information systems and routine surveillance practices. The results of this review suggest that electronic surveillance systems should be developed to maximize the efficacy of abundant electronic data sources existing within hospitals.

Automated surveillance for healthcare-associated infections: opportunities for improvement

Surveillance of healthcare-associated infections is a cornerstone of infection prevention programs, and reporting of infection rates is increasingly required. Traditionally, surveillance is based on manual medical records review; however, this is very labor intensive and vulnerable to misclassification. Existing electronic surveillance systems based on classification algorithms using microbiology results, antibiotic use data, and/or discharge codes have increased the efficiency and completeness of surveillance by preselecting high-risk patients for manual review. However, shifting to electronic surveillance using multivariable prediction models based on available clinical patient data will allow for even more efficient detection of infection. With ongoing developments in healthcare information technology, implementation of the latter surveillance systems will become increasingly feasible. As with current predominantly manual methods, several challenges remain, such as completeness of postdischarge surveillance and adequate adjustment for underlying patient characteristics, especially for comparison of healthcare-associated infection rates across institutions.

Utilization of electronic medical records to build a detection model for surveillance of healthcare-associated urinary tract infections

In this study, we propose an approach to build a detection model for surveillance of healthcare-associated urinary tract infection (HA-UTI) based on the variables extracted from the electronic medical records (EMRs) in a 730-bed, tertiary-care teaching hospital in Taiwan. Firstly we mapped the CDC's HA-UTI case definitions to a set of variables, and identified the variables whose values could be derived from the EMRs of the hospital automatically. Then with these variables we performed discriminant analysis (DA) on a training set of the EMRs to construct a discriminant function (DF) for the classification of a patient with or without HA-UTI. Finally, we evaluated the sensitivity, specificity, and overall accuracy of the function using a testing set of EMRs. In this study, six surveillance variables (fever, urine culture, blood culture, routine urinalysis, antibiotic use, and invasive devices) were identified whose values could be derived from the EMRs of the hospital. The sensitivity, specificity and overall accuracy of the built DF were 100 %, 94.61 %, and 94.65 %, respectively. Since most hospitals may adopt their EMRs piece-by-piece to meet their functional requirements, the variables that are available in the EMRs may differ. Our approach can build a detection model with these variables to achieve a high sensitivity, specificity and accuracy for automatically detecting suspected HA-UTI cases. Therefore, our approach on one hand can reduce the efforts in building the model; on the other hand, can facilitate adoption of EMRs for HAI surveillance and control.

Automated detection of healthcare associated infections: external validation and updating of a model for surveillance of drain-related meningitis

OBJECTIVE

Automated surveillance of healthcare-associated infections can improve efficiency and reliability of surveillance. The aim was to validate and update a previously developed multivariable prediction model for the detection of drain-related meningitis (DRM).

DESIGN

Retrospective cohort study using traditional surveillance by infection control professionals as reference standard.

PATIENTS

Patients receiving an external cerebrospinal fluid drain, either ventricular (EVD) or lumbar (ELD) in a tertiary medical care center. Children, patients with simultaneous drains, <1 day of follow-up or pre-existing meningitis were excluded leaving 105 patients in validation set (2010-2011) and 653 in updating set (2004-2011).

METHODS

For validation, the original model was applied. Discrimination, classification and calibration were assessed. For updating, data from all available years was used to optimally re-estimate coefficients and determine whether extension with new predictors is necessary. The updated model was validated and adjusted for optimism (overfitting) using bootstrapping techniques.

RESULTS

In model validation, the rate of DRM was 17.4/1000 days at risk. All cases were detected by the model. The area under the ROC curve was 0.951. The positive predictive value was 58.8% (95% CI 40.7-75.4) and calibration was good. The revised model also includes Gram stain results. Area under the ROC curve after correction for optimism was 0.963 (95% CI 0.953- 0.974). Group-level prediction was adequate.

CONCLUSIONS

The previously developed multivariable prediction model maintains discriminatory power and calibration in an independent patient population. The updated model incorporates all available data and performs well, also after elaborate adjustment for optimism.

Classification of positive blood cultures: computer algorithms versus physicians' assessment--development of tools for surveillance of bloodstream infection prognosis using population-based laboratory databases

Background

Information from blood cultures is utilized for infection control, public health surveillance, and clinical outcome research. This information can be enriched by physicians’ assessments of positive blood cultures, which are, however, often available from selected patient groups or pathogens only. The aim of this work was to determine whether patients with positive blood cultures can be classified effectively for outcome research in epidemiological studies by the use of administrative data and computer algorithms, taking physicians’ assessments as reference.

Methods

Physicians’ assessments of positive blood cultures were routinely recorded at two Danish hospitals from 2006 through 2008. The physicians’ assessments classified positive blood cultures as: a) contamination or bloodstream infection; b) bloodstream infection as mono- or polymicrobial; c) bloodstream infection as community- or hospital-onset; d) community-onset bloodstream infection as healthcare-associated or not. We applied the computer algorithms to data from laboratory databases and the Danish National Patient Registry to classify the same groups and compared these with the physicians’ assessments as reference episodes. For each classification, we tabulated episodes derived by the physicians’ assessment and the computer algorithm and compared 30-day mortality between concordant and discrepant groups with adjustment for age, gender, and comorbidity.

Results

Physicians derived 9,482 reference episodes from 21,705 positive blood cultures. The agreement between computer algorithms and physicians’ assessments was high for contamination vs. bloodstream infection (8,966/9,482 reference episodes [96.6%], Kappa = 0.83) and mono- vs. polymicrobial bloodstream infection (6,932/7,288 reference episodes [95.2%], Kappa = 0.76), but lower for community- vs. hospital-onset bloodstream infection (6,056/7,288 reference episodes [83.1%], Kappa = 0.57) and healthcare-association (3,032/4,740 reference episodes [64.0%], Kappa = 0.15). The 30-day mortality in the discrepant groups differed from the concordant groups as regards community- vs. hospital-onset, whereas there were no material differences within the other comparison groups.

Conclusions

Using data from health administrative registries, we found high agreement between the computer algorithms and the physicians’ assessments as regards contamination vs. bloodstream infection and monomicrobial vs. polymicrobial bloodstream infection, whereas there was only moderate agreement between the computer algorithms and the physicians’ assessments concerning the place of onset. These results provide new information on the utility of computer algorithms derived from health administrative registries.

Bundling hand hygiene interventions and measurement to decrease health care-associated infections

Proper performance of hand hygiene at key moments during patient care is the most important means of preventing health care-associated infections (HAIs). With increasing awareness of the cost and societal impact caused by HAIs has come the realization that hand hygiene improvement initiatives are crucial to reducing the burden of HAIs. Multimodal strategies have emerged as the best approach to improving hand hygiene compliance. These strategies use a variety of intervention components intended to address obstacles to complying with good hand hygiene practices, and to reinforce behavioral change. Although research has substantiated the effectiveness of the multimodal design, challenges remain in promoting widespread adoption and implementation of a coordinated approach. This article reviews elements of a multimodal approach to improve hand hygiene and advocates the use of a "bundled" strategy. Eight key components of this bundle are proposed as a cohesive program to enable the deployment of synergistic, coordinated efforts to promote good hand hygiene practice. A consistent, bundled methodology implemented at multiple study centers would standardize processes and allow comparison of outcomes, validation of the methodology, and benchmarking. Most important, a bundled approach can lead to sustained infection reduction.

Automated detection of external ventricular and lumbar drain-related meningitis using laboratory and microbiology results and medication data

Objective

Monitoring of healthcare-associated infection rates is important for infection control and hospital benchmarking. However, manual surveillance is time-consuming and susceptible to error. The aim was, therefore, to develop a prediction model to retrospectively detect drain-related meningitis (DRM), a frequently occurring nosocomial infection, using routinely collected data from a clinical data warehouse.

Methods

As part of the hospital infection control program, all patients receiving an external ventricular (EVD) or lumbar drain (ELD) (2004 to 2009; n = 742) had been evaluated for the development of DRM through chart review and standardized diagnostic criteria by infection control staff; this was the reference standard. Children, patients dying <24 hours after drain insertion or with <1 day follow-up and patients with infection at the time of insertion or multiple simultaneous drains were excluded. Logistic regression was used to develop a model predicting the occurrence of DRM. Missing data were imputed using multiple imputation. Bootstrapping was applied to increase generalizability.

Results

537 patients remained after application of exclusion criteria, of which 82 developed DRM (13.5/1000 days at risk). The automated model to detect DRM included the number of drains placed, drain type, blood leukocyte count, C-reactive protein, cerebrospinal fluid leukocyte count and culture result, number of antibiotics started during admission, and empiric antibiotic therapy. Discriminatory power of this model was excellent (area under the ROC curve 0.97). The model achieved 98.8% sensitivity (95% CI 88.0% to 99.9%) and specificity of 87.9% (84.6% to 90.8%). Positive and negative predictive values were 56.9% (50.8% to 67.9%) and 99.9% (98.6% to 99.9%), respectively. Predicted yearly infection rates concurred with observed infection rates.

Conclusion

A prediction model based on multi-source data stored in a clinical data warehouse could accurately quantify rates of DRM. Automated detection using this statistical approach is feasible and could be applied to other nosocomial infections.

Automated detection of nosocomial infections: evaluation of different strategies in an intensive care unit 2000-2006

The aim of this study was to evaluate seven different strategies for the automated detection of nosocomial infections (NIs) in an intensive care unit (ICU) by using different hospital information systems: microbiology database, antibiotic prescriptions, medico-administrative database, and textual hospital discharge summaries. The study involved 1,499 patients admitted to an ICU of the University Hospital of Lyon (France) between 2000 and 2006. The data were extracted from the microbiology laboratory information system, the clinical information system on the ward and the medico-administrative database. Different algorithms and strategies were developed, using these data sources individually or in combination. The performances of each strategy were assessed by comparing the results with the ward data collected as a national standardised surveillance protocol, adapted from the National Nosocomial Infections Surveillance system as the gold standard. From 1,499 patients, 282 NIs were reported. The strategy with the best sensitivity for detecting these infections using an automated method was the combination of antibiotic prescription or microbiology, with a sensitivity of 99.3% [95% confidence interval (CI): 98.2-100] and a specificity of 56.8% (95% CI: 54.0-59.6). Automated methods of NI detection represent an alternative to traditional monitoring methods. Further study involving more ICUs should be performed before national recommendations can be established.

Surveillance of selected post-caesarean infections based on electronic registries: validation study including post-discharge infections

The importance of surveillance of post-discharge infections has increased as a consequence of shorter hospital stay after surgical procedures. This study examined the ability of a computer-based surveillance system to identify urinary tract infections (UTIs) and postoperative wound infections (PWIs) within 30 days after caesarean section. We assessed the use of data from various electronic registries to identify patients with post-caesarean UTI and PWI classified according to a reference standard. The standard was based on information from medical records and self-reported data (questionnaire) using modified Centers for Disease Control and Prevention definitions. The sensitivity of the computer system in detecting UTI diagnosed during hospital stay, readmission or at visits to hospital outpatient clinics was 80.0%; the specificity was 99.9%. For post-discharge UTIs diagnosed outside the hospital, sensitivity and specificity were 76.3% and 99.9%, respectively. For PWIs diagnosed in hospital and post-discharge outside hospital, sensitivities were 77.1% and 68.9%, and the specificities 99.5% and 98.2%. We conclude that a computer-based surveillance system may identify in-hospital infections and post-discharge infections with a relatively high sensitivity and excellent specificity.

Use of anti-infective serial prevalence studies to identify and monitor hospital-acquired infection

We developed the 'Pragmatic Proxy Protocol' (PPP) to estimate the prevalence of hospital-acquired infection (HAI) by integrating our existing pharmacy serial point prevalence studies of anti-infective prescribing practices with electronic data on microbiological and radiographic markers of infection. Our method was evaluated against the standard Hospital Infection Society/Infection Control Nurses Association Protocol (HIP). In the non-surgical patients, PPP has a sensitivity of 1.00 [confidence interval (CI): 0.70-1.00] and specificity of 0.97 (CI: 0.93-0.99). PPP suggests that for non-surgical patients, the prevalence of HAI using HIP could be underestimated by 42%. PPP takes about two-thirds of the time of HIP (75 vs 106 h) and is at least one-third cheaper. It could easily be adapted to advances in electronic reporting and, with the development of Anti-infective Care Bundles, would increase its sensitivity for the detection of HAI in surgical patients. PPP could be used to increase the frequency of routine HAI surveillance to determine the overall burden of infection and assess the efficacy of intervention strategies in a timely manner allowing rapid, direct feedback and engagement with clinicians.

Automated surveillance for central line-associated bloodstream infection in intensive care units

OBJECTIVE

To develop and evaluate computer algorithms with high negative predictive values that augment traditional surveillance for central line-associated bloodstream infection (CLABSI).

SETTING

Barnes-Jewish Hospital, a 1,250-bed tertiary care academic hospital in Saint Louis, Missouri.

METHODS

We evaluated all adult patients in intensive care units who had blood samples collected during the period from July 1, 2005, to June 30, 2006, that were positive for a recognized pathogen on culture. Each isolate recovered from culture was evaluated using the definitions for nosocomial CLABSI provided by the National Healthcare Safety Network of the Centers for Disease Control and Prevention. Using manual surveillance by infection prevention specialists as the gold standard, we assessed the ability of various combinations of dichotomous rules to determine whether an isolate was associated with a CLABSI. Sensitivity, specificity, and predictive values were calculated.

RESULTS

Infection prevention specialists identified 67 cases of CLABSI associated with 771 isolates recovered from blood samples. The algorithms excluded approximately 40%-62% of the isolates from consideration as possible causes of CLABSI. The simplest algorithm, with 2 dichotomous rules (ie, the collection of blood samples more than 48 hours after admission and the presence of a central venous catheter within 48 hours before collection of blood samples), had the highest negative predictive value (99.4%) and the lowest specificity (44.2%) for CLABSI. Augmentation of this algorithm with rules for common skin contaminants confirmed by another positive blood culture result yielded in a negative predictive value of 99.2% and a specificity of 68.0%.

CONCLUSIONS

An automated approach to surveillance for CLABSI that is characterized by a high negative predictive value can accurately identify and exclude positive culture results not representing CLABSI from further manual surveillance.

Validity of electronic surveillance systems: a systematic review

Electronic surveillance that utilises information held in databases is more efficient than conventional infection surveillance methods. Validity is not well-defined, however. We systematically reviewed studies comparing the utility of electronic and conventional surveillance methods. Publications were identified using Medline (1980-2007) and bibliographic review. The sensitivity and specificity of electronic compared with conventional surveillance was reported. Twenty-four studies were included. Six studies reported that nosocomial infections could be detected utilising microbiology data alone with good overall sensitivity (range: 63-91%) and excellent specificity (range: 87 to >99%). Two studies used three laboratory-based algorithms for the detection of infection outbreaks yielding variable utility measures (sensitivity, range: 43-91%; specificity, range: 67-86%). Seven studies using only administrative data including discharge coding (International Classification of Diseases, 9th edn, Clinical Modification) and pharmacy data claimed databases had good sensitivity (range: 59-96%) and excellent specificity (range: 95 to >99%) in detecting nosocomial infections. Six studies combined both laboratory and administrative data for a range of infections, and overall had higher sensitivity (range: 71-94%) but lower specificity (range: 47 to >99%) than with use of either alone. Three studies evaluated community-acquired infections with variable results. Electronic surveillance has moderate to excellent utility compared with conventional methods for nosocomial infections. Future studies are needed to refine electronic algorithms further, especially with community-onset infections.

Economics of infection control surveillance technology: cost-effective or just cost?

BACKGROUND

Previous studies have suggested that informatics tools, such as automated alert and decision support systems, may increase the efficiency and quality of infection control surveillance. However, little is known about the cost-effectiveness of these tools.

METHODS

We focus on 2 types of economic analyses that have utility in assessing infection control interventions (cost-effectiveness analysis and business-case analysis) and review the available literature on the economics of computerized infection control surveillance systems.

RESULTS

Previous studies on the effectiveness of computerized infection control surveillance have been limited to assessments of whether these tools increase the sensitivity and specificity of surveillance over traditional methods. Furthermore, we identified only 2 studies that assessed the costs associated with computerized infection control surveillance. Thus, it remains unknown whether computerized infection control surveillance systems are cost-effective and whether use of these systems improves patient outcomes.

CONCLUSION

The existing data are insufficient to allow for a summary conclusion on the cost-effectiveness of infection control surveillance technology. All future studies of computerized infection control surveillance systems should aim to collect outcomes and economic data to inform decision making and assist hospitals with completing business-cases analyses.

Quality of information: a European challenge

Since the end of the 1970s, many countries have started to set up programmes to control healthcare-associated infections (HAIs) and to achieve a safe and sustainable development of their healthcare facilities that minimises the risk of infection. Surveillance is a usual component of any organised programme to address the problem either at national, regional or local level. So a considerable effort has been made by the national Public Health Authorities of EU Member States to foster and extend the surveillance of HAI via the production of increasingly standardised indicators. This information is used by Infection Control teams to implement preventive strategies, to evaluate the magnitude of the problem and to understand variations in the risks of HAI. At the same time, Public Health authorities and healthcare financing agencies in several countries have attempted to generalise the production of such indicators at an official level and use them as a global approach for hospital quality assessment, accreditation, continuous quality improvement and communication with patients and the general population.