Comparison of automated strategies for surveillance of nosocomial bacteremia

Systematic scoping review of automated systems for the surveillance of healthcare-associated bloodstream infections related to intravascular catheters

INTRODUCTION

Intravascular catheters are crucial devices in medical practice that increase the risk of healthcare-associated infections (HAIs), and related health-economic adverse outcomes. This scoping review aims to provide a comprehensive overview of published automated algorithms for surveillance of catheter-related bloodstream infections (CRBSI) and central line-associated bloodstream infections (CLABSI).

METHODS

We performed a scoping review based on a systematic search of the literature in PubMed and EMBASE from 1 January 2000 to 31 December 2021. Studies were included if they evaluated predictive performance of automated surveillance algorithms for CLABSI/CRBSI detection and used manually collected surveillance data as reference. We assessed the design of the automated systems, including the definitions used to develop algorithms (CLABSI versus CRBSI), the datasets and denominators used, and the algorithms evaluated in each of the studies.

RESULTS

We screened 586 studies based on title and abstract, and 99 were assessed based on full text. Nine studies were included in the scoping review. Most studies were monocentric (n = 5), and they identified CLABSI (n = 7) as an outcome. The majority of the studies used administrative and microbiological data (n = 9) and five studies included the presence of a vascular central line in their automated system. Six studies explained the denominator they selected, five of which chose central line-days. The most common rules and steps used in the algorithms were categorized as hospital-acquired rules, infection rules (infection versus contamination), deduplication, episode grouping, secondary BSI rules (secondary versus primary BSI), and catheter-associated rules.

CONCLUSION

The automated surveillance systems that we identified were heterogeneous in terms of definitions, datasets and denominators used, with a combination of rules in each algorithm. Further guidelines and studies are needed to develop and implement algorithms to detect CLABSI/CRBSI, with standardized definitions, appropriate data sources and suitable denominators.

The accuracy of fully-automated algorithms for the surveillance of central venous catheter-related bloodstream infection in hospitalised patients

BACKGROUND

Continuous surveillance for healthcare-associated infections such as central venous catheter-related bloodstream infections (CVC-BSI) is crucial for prevention. However, traditional surveillance methods are resource-intensive and prone to bias. This study aimed to develop and validate fully-automated surveillance algorithms for CVC-BSI.

METHODS

Two algorithms were developed using electronic health record data from 1000 admissions with a positive blood culture (BCx) at Karolinska University Hospital from 2017: (1) Combining microbiological findings in BCx and CVC cultures with BSI symptoms; (2) Only using microbiological findings. These algorithms were validated in 5170 potential CVC-BSI-episodes from all admissions in 2018-2019, and results extrapolated to all potential CVC-BSI-episodes within this period (n = 181,354). The reference standard was manual record review according to ECDC's definition of microbiologically confirmed CVC-BSI (CRI3-CVC).

RESULTS

In the potential CVC-BSI-episodes, 51 fulfilled ECDC's definition and the algorithms identified 47 and 49 episodes as CVC-BSI, respectively. Both algorithms performed well in assessing CVC-BSI. Overall, algorithm 2 performed slightly better with in the total period a sensitivity of 0.880 (95%-CI 0.783-0.959), specificity of 1.000 (95%-CI 0.999-1.000), PPV of 0.918 (95%-CI 0.833-0.981) and NPV of 1.000 (95%-CI 0.999-1.000). Incidence according to the reference and algorithm 2 was 0.33 and 0.31 per 1000 in-patient hospital-days, respectively.

CONCLUSIONS

Both fully-automated surveillance algorithms for CVC-BSI performed well and could effectively replace manual surveillance. The simpler algorithm, using only microbiology data, is suitable when BCx testing adheres to recommendations, otherwise the algorithm using symptom data might be required. Further validation in other settings is necessary to assess the algorithms' generalisability.

Predictive performance of automated surveillance algorithms for intravascular catheter bloodstream infections: a systematic review and meta-analysis

BACKGROUND

Intravascular catheter infections are associated with adverse clinical outcomes. However, a significant proportion of these infections are preventable. Evaluations of the performance of automated surveillance systems for adequate monitoring of central-line associated bloodstream infection (CLABSI) or catheter-related bloodstream infection (CRBSI) are limited.

OBJECTIVES

We evaluated the predictive performance of automated algorithms for CLABSI/CRBSI detection, and investigated which parameters included in automated algorithms provide the greatest accuracy for CLABSI/CRBSI detection.

METHODS

We performed a meta-analysis based on a systematic search of published studies in PubMed and EMBASE from 1 January 2000 to 31 December 2021. We included studies that evaluated predictive performance of automated surveillance algorithms for CLABSI/CRBSI detection and used manually collected surveillance data as reference. We estimated the pooled sensitivity and specificity of algorithms for accuracy and performed a univariable meta-regression of the different parameters used across algorithms.

RESULTS

The search identified five full text studies and 32 different algorithms or study populations were included in the meta-analysis. All studies analysed central venous catheters and identified CLABSI or CRBSI as an outcome. Pooled sensitivity and specificity of automated surveillance algorithm were 0.88 [95%CI 0.84-0.91] and 0.86 [95%CI 0.79-0.92] with significant heterogeneity (I2 = 91.9, p < 0.001 and I2 = 99.2, p < 0.001, respectively). In meta-regression, algorithms that include results of microbiological cultures from specific specimens (respiratory, urine and wound) to exclude non-CRBSI had higher specificity estimates (0.92, 95%CI 0.88-0.96) than algorithms that include results of microbiological cultures from any other body sites (0.88, 95% CI 0.81-0.95). The addition of clinical signs as a predictor did not improve performance of these algorithms with similar specificity estimates (0.92, 95%CI 0.88-0.96).

CONCLUSIONS

Performance of automated algorithms for detection of intravascular catheter infections in comparison to manual surveillance seems encouraging. The development of automated algorithms should consider the inclusion of results of microbiological cultures from specific specimens to exclude non-CRBSI, while the inclusion of clinical data may not have an added-value. Trail Registration Prospectively registered with International prospective register of systematic reviews (PROSPERO ID CRD42022299641; January 21, 2022). https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42022299641.

Development and validation of MicrobEx: an open-source package for microbiology culture concept extraction

Objective

Microbiology culture reports contain critical information for important clinical and public health applications. However, microbiology reports often have complex, semistructured, free-text data that present a barrier for secondary use. Here we present the development and validation of an open-source package designed to ingest free-text microbiology reports, determine whether the culture is positive, and return a list of Systemized Nomenclature of Medicine (SNOMED)-CT mapped bacteria.

Materials and Methods

Our concept extraction Python package, MicrobEx, is built upon a rule-based natural language processing algorithm and was developed using microbiology reports from 2 different electronic health record systems in a large healthcare organization, and then externally validated on the reports of 2 other institutions with manually reviewed results as a benchmark.

Results

MicrobEx achieved F1 scores &gt;0.95 on all classification tasks across 2 independent validation sets with minimal customization. Additionally, MicrobEx matched or surpassed our MetaMap-based benchmark algorithm performance across positive culture classification and species capture classification tasks.

Discussion

Our results suggest that MicrobEx can be used to reliably estimate binary bacterial culture status, extract bacterial species, and map these to SNOMED organism observations when applied to semistructured, free-text microbiology reports from different institutions with relatively low customization.

Conclusion

MicrobEx offers an open-source software solution (available on both GitHub and PyPI) for bacterial culture status estimation and bacterial species extraction from free-text microbiology reports. The package was designed to be reused and adapted to individual institutions as an upstream process for other clinical applications such as: machine learning, clinical decision support, and disease surveillance systems.

Electronically assisted surveillance systems of healthcare-associated infections: a systematic review

Background

Surveillance of healthcare-associated infections (HAI) is the basis of each infection control programme and, in case of acute care hospitals, should ideally include all hospital wards, medical specialties as well as all types of HAI. Traditional surveillance is labour intensive and electronically assisted surveillance systems (EASS) hold the promise to increase efficiency.

Objectives

To give insight in the performance characteristics of different approaches to EASS and the quality of the studies designed to evaluate them.

Methods

In this systematic review, online databases were searched and studies that compared an EASS with a traditional surveillance method were included. Two different indicators were extracted from each study, one regarding the quality of design (including reporting efficiency) and one based on the performance (e.g. specificity and sensitivity) of the EASS presented.

Results

A total of 78 studies were included. The majority of EASS (n = 72) consisted of an algorithm-based selection step followed by confirmatory assessment. The algorithms used different sets of variables. Only a minority (n = 7) of EASS were hospital-wide and designed to detect all types of HAI. Sensitivity of EASS was generally high (> 0.8), but specificity varied (0.37–1). Less than 20% (n = 14) of the studies presented data on the efficiency gains achieved.

Conclusions

Electronically assisted surveillance of HAI has yet to reach a mature stage and to be used routinely in healthcare settings. We recommend that future studies on the development and implementation of EASS of HAI focus on thorough validation, reproducibility, standardised datasets and detailed information on efficiency.

Protocol for developing a surveillance system for surgical site infections

Purpose

Healthcare-associated infections (HCAIs/ HAIs) are the most common adverse occurrences during health care delivery. Across the globe, millions of patients are affected by HAIs annually, with a higher burden and impact in developing nations. a major lacuna in planning preventing protocols is the absence of National Surveillance Systems in most low-middle income countries, which also prevents allocation of resources to the high-priority areas. Among all the HAIs, there is a huge global burden of SSIs, in terms of morbidity, prolonged hospital stays, increased antimicrobial treatment as well as attributable mortality.

Method

This manuscript details the process of establishment of an SSI surveillance protocol at a level-1 trauma centre in North India.

Result and Conclusion

Surveillance is an essential tool to reduce this burden. It is also an important primary step in recognizing problems and priorities, and it plays a crucial role in identifying risk factors for SSI and to be able to target modifiable risk factors. Therefore, it is imperative to establish reliable systems for surveillance of HAIs, to regularly estimate the actual burden of HAIs, and to use these data for developing indigenous preventive measures, tailored to the country's priorities.

WMSS: A Web-Based Multitiered Surveillance System for Predicting CLABSI

Central-line-associated bloodstream infection (CLABSI) rates are a key quality metric for comparing hospital quality and safety. Manual surveillance systems for CLABSIs are time-consuming and often limited to intensive care units (ICUs). A computer-automated method of CLABSI detection can improve the validity of surveillance. A new web-based, multitiered surveillance system for predicting and reducing CLABSI is proposed. The system has the capability to collect patient-related data from hospital databases and hence predict the patient infection automatically based on knowledge discovery rules and CLABSI decision standard algorithms. In addition, the system has a built-in simulator for generating patients' data records, when needed, offering the capability to train nurses and medical staff for enhancing their qualifications. Applying the proposed system, both CLABSI rates and patient treatment costs can be reduced significantly. The system has many benefits, among which there is the following: it is a web-based system that can collect real patients' data from many IT resources using iPhone, iPad, laptops, Internet, scanners, and hospital databases. These facilities help to collect patients' actual data quickly and safely in electronic format and hence predict CLABSI efficiently. Automation of the patients' data diagnosis process helps in reducing CLABSI detection times. The system is multimedia-based; it uses text, colors, and graphics to enhance patient healthcare report generation and charts. It helps healthcare decision makers to review and approve policies and surveillance plans to reduce and prevent CLABSI.

A Web-Based, Hospital-Wide Health Care-Associated Bloodstream Infection Surveillance and Classification System: Development and Evaluation

BACKGROUND

Surveillance of health care-associated infections is an essential component of infection prevention programs, but conventional systems are labor intensive and performance dependent.

OBJECTIVE

To develop an automatic surveillance and classification system for health care-associated bloodstream infection (HABSI), and to evaluate its performance by comparing it with a conventional infection control personnel (ICP)-based surveillance system.

METHODS

We developed a Web-based system that was integrated into the medical information system of a 2200-bed teaching hospital in Taiwan. The system automatically detects and classifies HABSIs.

RESULTS

In this study, the number of computer-detected HABSIs correlated closely with the number of HABSIs detected by ICP by department (n=20; r=.999 P<.001) and by time (n=14; r=.941; P<.001). Compared with reference standards, this system performed excellently with regard to sensitivity (98.16%), specificity (99.96%), positive predictive value (95.81%), and negative predictive value (99.98%). The system enabled decreasing the delay in confirmation of HABSI cases, on average, by 29 days.

CONCLUSIONS

This system provides reliable and objective HABSI data for quality indicators, improving the delay caused by a conventional surveillance system.

Accuracy of using automated methods for detecting adverse events from electronic health record data: a research protocol

Background

Adverse events are associated with significant morbidity, mortality and cost in hospitalized patients. Measuring adverse events is necessary for quality improvement, but current detection methods are inaccurate, untimely and expensive. The advent of electronic health records and the development of automated methods for encoding and classifying electronic narrative data, such as natural language processing, offer an opportunity to identify potentially better methods. The objective of this study is to determine the accuracy of using automated methods for detecting three highly prevalent adverse events: a) hospital-acquired pneumonia, b) catheter-associated bloodstream infections, and c) in-hospital falls.

Methods/design

This validation study will be conducted at two large Canadian academic health centres: the McGill University Health Centre (MUHC) and The Ottawa Hospital (TOH). The study population consists of all medical, surgical and intensive care unit patients admitted to these centres between 2008 and 2014. An automated detection algorithm will be developed and validated for each of the three adverse events using electronic data extracted from multiple clinical databases. A random sample of MUHC patients will be used to develop the automated detection algorithms (cohort 1, development set). The accuracy of these algorithms will be assessed using chart review as the reference standard. Then, receiver operating characteristic curves will be used to identify optimal cut points for each of the data sources. Multivariate logistic regression and the areas under curve (AUC) will be used to identify the optimal combination of data sources that maximize the accuracy of adverse event detection. The most accurate algorithms will then be validated on a second random sample of MUHC patients (cohort 1, validation set), and accuracy will be measured using chart review as the reference standard. The most accurate algorithms validated at the MUHC will then be applied to TOH data (cohort 2), and their accuracy will be assessed using a reference standard assessment of the medical chart.

Discussion

There is a need for more accurate, timely and efficient measures of adverse events in acute care hospitals. This is a critical requirement for evaluating the effectiveness of preventive interventions and for tracking progress in patient safety through time.

Data use and effectiveness in electronic surveillance of healthcare associated infections in the 21st century: a systematic review

OBJECTIVE

As more electronic health records have become available during the last decade, we aimed to uncover recent trends in use of electronically available patient data by electronic surveillance systems for healthcare associated infections (HAIs) and identify consequences for system effectiveness.

METHODS

A systematic review of published literature evaluating electronic HAI surveillance systems was performed. The PubMed service was used to retrieve publications between January 2001 and December 2011. Studies were included in the review if they accurately described what electronic data were used and if system effectiveness was evaluated using sensitivity, specificity, positive predictive value, or negative predictive value. Trends were identified by analyzing changes in the number and types of electronic data sources used.

RESULTS

26 publications comprising discussions on 27 electronic systems met the eligibility criteria. Trend analysis showed that systems use an increasing number of data sources which are either medico-administrative or clinical and laboratory-based data. Trends on the use of individual types of electronic data confirmed the paramount role of microbiology data in HAI detection, but also showed increased use of biochemistry and pharmacy data, and the limited adoption of clinical data and physician narratives. System effectiveness assessments indicate that the use of heterogeneous data sources results in higher system sensitivity at the expense of specificity.

CONCLUSIONS

Driven by the increased availability of electronic patient data, electronic HAI surveillance systems use more data, making systems more sensitive yet less specific, but also allow systems to be tailored to the needs of healthcare institutes' surveillance programs.

Public health surveillance in the dialysis setting: opportunities and challenges for using electronic health records

The US Centers for Disease Control and Prevention has conducted public health surveillance for healthcare-associated infections (HAIs) in dialysis facilities since the 1970s, evolving from facility-level surveys to patient-level surveillance systems. The Centers for Medicare and Medicaid Services (CMS) recently implemented incentives for all end-stage renal disease (ESRD) facilities to monitor and report patient-level quality indicators to the Centers for Disease Control and Prevention's (CDC's) National Healthcare Safety Network (NHSN) in accordance with the NHSN Dialysis Event Protocol. These CMS incentives have led to a rapid increase in dialysis facility NHSN enrollment during 2012. Ongoing challenges to HAI surveillance in this setting include variability in the surveillance process, assurance of data quality, and staff time and resource requirements. Use of existing electronic health records (EHR), especially in conjunction with detection algorithms, has increasingly been shown to produce valid and reliable estimates of HAI frequency in acute care hospitals. Given the large number of dialysis facilities that are now beginning to conduct surveillance using NHSN, the typical lack of dedicated infection prevention personnel in those facilities, and the widespread use of EHR in large dialysis provider organizations, the use of EHR will probably become a cornerstone of surveillance in these settings. Implemented properly, the use of EHR to support public health surveillance has enormous potential to focus and strengthen infection prevention activities in dialysis facilities. Systematic, ongoing validation efforts will be vital to ensure that reported data are accurate, permit valid comparisons of facility performance, and effectively support improved outcomes for dialysis patients.

Diagnostic performance of electronic syndromic surveillance systems in acute care: a systematic review

CONTEXT

Healthcare Electronic Syndromic Surveillance (ESS) is the systematic collection, analysis and interpretation of ongoing clinical data with subsequent dissemination of results, which aid clinical decision-making.

OBJECTIVE

To evaluate, classify and analyze the diagnostic performance, strengths and limitations of existing acute care ESS systems.

DATA SOURCES

All available to us studies in Ovid MEDLINE, Ovid EMBASE, CINAHL and Scopus databases, from as early as January 1972 through the first week of September 2012.

STUDY SELECTION

Prospective and retrospective trials, examining the diagnostic performance of inpatient ESS and providing objective diagnostic data including sensitivity, specificity, positive and negative predictive values.

DATA EXTRACTION

Two independent reviewers extracted diagnostic performance data on ESS systems, including clinical area, number of decision points, sensitivity and specificity. Positive and negative likelihood ratios were calculated for each healthcare ESS system. A likelihood matrix summarizing the various ESS systems performance was created.

RESULTS

The described search strategy yielded 1639 articles. Of these, 1497 were excluded on abstract information. After full text review, abstraction and arbitration with a third reviewer, 33 studies met inclusion criteria, reporting 102,611 ESS decision points. The yielded I2 was high (98.8%), precluding meta-analysis. Performance was variable, with sensitivities ranging from 21% -100% and specificities ranging from 5%-100%.

CONCLUSIONS

There is significant heterogeneity in the diagnostic performance of the available ESS implements in acute care, stemming from the wide spectrum of different clinical entities and ESS systems. Based on the results, we introduce a conceptual framework using a likelihood ratio matrix for evaluation and meaningful application of future, frontline clinical decision support systems.

Advances in electronic surveillance for healthcare-associated infections in the 21st Century: a systematic review

BACKGROUND

Traditional methodologies for healthcare-associated infection (HCAI) surveillance can be resource intensive and time consuming. As a consequence, surveillance is often limited to specific organisms or conditions. Various electronic databases exist within the healthcare setting and may be utilized to perform HCAI surveillance.

AIM

To assess the utility of electronic surveillance systems for monitoring and detecting HCAI.

METHODS

A systematic review of published literature on surveillance of HCAI was performed. Databases were searched for studies published between January 2000 and December 2011. Search terms were divided into infection, surveillance and data management terms, and combined using Boolean operators. Studies were included for review if they demonstrated or proposed the use of electronic systems for HCAI surveillance.

FINDINGS

In total, 44 studies met the inclusion criteria. For the majority of studies, emphasis was on the linkage of electronic databases to provide automated methods for monitoring infections in specific clinical settings. Twenty-one studies assessed the performance of their method with traditional surveillance methodologies or a manual reference method. Where sensitivity and specificity were calculated, these varied depending on the organism or condition being surveyed and the data sources employed.

CONCLUSIONS

The implementation of electronic surveillance was found to be feasible in many settings, with several systems fully integrated into hospital information systems and routine surveillance practices. The results of this review suggest that electronic surveillance systems should be developed to maximize the efficacy of abundant electronic data sources existing within hospitals.

Classification of positive blood cultures: computer algorithms versus physicians' assessment--development of tools for surveillance of bloodstream infection prognosis using population-based laboratory databases

Background

Information from blood cultures is utilized for infection control, public health surveillance, and clinical outcome research. This information can be enriched by physicians’ assessments of positive blood cultures, which are, however, often available from selected patient groups or pathogens only. The aim of this work was to determine whether patients with positive blood cultures can be classified effectively for outcome research in epidemiological studies by the use of administrative data and computer algorithms, taking physicians’ assessments as reference.

Methods

Physicians’ assessments of positive blood cultures were routinely recorded at two Danish hospitals from 2006 through 2008. The physicians’ assessments classified positive blood cultures as: a) contamination or bloodstream infection; b) bloodstream infection as mono- or polymicrobial; c) bloodstream infection as community- or hospital-onset; d) community-onset bloodstream infection as healthcare-associated or not. We applied the computer algorithms to data from laboratory databases and the Danish National Patient Registry to classify the same groups and compared these with the physicians’ assessments as reference episodes. For each classification, we tabulated episodes derived by the physicians’ assessment and the computer algorithm and compared 30-day mortality between concordant and discrepant groups with adjustment for age, gender, and comorbidity.

Results

Physicians derived 9,482 reference episodes from 21,705 positive blood cultures. The agreement between computer algorithms and physicians’ assessments was high for contamination vs. bloodstream infection (8,966/9,482 reference episodes [96.6%], Kappa = 0.83) and mono- vs. polymicrobial bloodstream infection (6,932/7,288 reference episodes [95.2%], Kappa = 0.76), but lower for community- vs. hospital-onset bloodstream infection (6,056/7,288 reference episodes [83.1%], Kappa = 0.57) and healthcare-association (3,032/4,740 reference episodes [64.0%], Kappa = 0.15). The 30-day mortality in the discrepant groups differed from the concordant groups as regards community- vs. hospital-onset, whereas there were no material differences within the other comparison groups.

Conclusions

Using data from health administrative registries, we found high agreement between the computer algorithms and the physicians’ assessments as regards contamination vs. bloodstream infection and monomicrobial vs. polymicrobial bloodstream infection, whereas there was only moderate agreement between the computer algorithms and the physicians’ assessments concerning the place of onset. These results provide new information on the utility of computer algorithms derived from health administrative registries.

Documentation of contact precautions in an electronic health record

Contact precautions are implemented to reduce transmission of multidrug-resistant organisms but may also increase hospital costs and patient complications. The goal of this study was to determine the prevalence of documentation of contact precautions (provider orders and nursing flowsheet documentation) in an electronic health record. Orders and nursing documentation were simultaneously present for only 42.3% of patient rooms with contact precaution signs, and 17.8% of rooms with signs had neither orders nor nursing documentation.

Automated detection of external ventricular and lumbar drain-related meningitis using laboratory and microbiology results and medication data

Objective

Monitoring of healthcare-associated infection rates is important for infection control and hospital benchmarking. However, manual surveillance is time-consuming and susceptible to error. The aim was, therefore, to develop a prediction model to retrospectively detect drain-related meningitis (DRM), a frequently occurring nosocomial infection, using routinely collected data from a clinical data warehouse.

Methods

As part of the hospital infection control program, all patients receiving an external ventricular (EVD) or lumbar drain (ELD) (2004 to 2009; n = 742) had been evaluated for the development of DRM through chart review and standardized diagnostic criteria by infection control staff; this was the reference standard. Children, patients dying <24 hours after drain insertion or with <1 day follow-up and patients with infection at the time of insertion or multiple simultaneous drains were excluded. Logistic regression was used to develop a model predicting the occurrence of DRM. Missing data were imputed using multiple imputation. Bootstrapping was applied to increase generalizability.

Results

537 patients remained after application of exclusion criteria, of which 82 developed DRM (13.5/1000 days at risk). The automated model to detect DRM included the number of drains placed, drain type, blood leukocyte count, C-reactive protein, cerebrospinal fluid leukocyte count and culture result, number of antibiotics started during admission, and empiric antibiotic therapy. Discriminatory power of this model was excellent (area under the ROC curve 0.97). The model achieved 98.8% sensitivity (95% CI 88.0% to 99.9%) and specificity of 87.9% (84.6% to 90.8%). Positive and negative predictive values were 56.9% (50.8% to 67.9%) and 99.9% (98.6% to 99.9%), respectively. Predicted yearly infection rates concurred with observed infection rates.

Conclusion

A prediction model based on multi-source data stored in a clinical data warehouse could accurately quantify rates of DRM. Automated detection using this statistical approach is feasible and could be applied to other nosocomial infections.

Automated detection of nosocomial infections: evaluation of different strategies in an intensive care unit 2000-2006

The aim of this study was to evaluate seven different strategies for the automated detection of nosocomial infections (NIs) in an intensive care unit (ICU) by using different hospital information systems: microbiology database, antibiotic prescriptions, medico-administrative database, and textual hospital discharge summaries. The study involved 1,499 patients admitted to an ICU of the University Hospital of Lyon (France) between 2000 and 2006. The data were extracted from the microbiology laboratory information system, the clinical information system on the ward and the medico-administrative database. Different algorithms and strategies were developed, using these data sources individually or in combination. The performances of each strategy were assessed by comparing the results with the ward data collected as a national standardised surveillance protocol, adapted from the National Nosocomial Infections Surveillance system as the gold standard. From 1,499 patients, 282 NIs were reported. The strategy with the best sensitivity for detecting these infections using an automated method was the combination of antibiotic prescription or microbiology, with a sensitivity of 99.3% [95% confidence interval (CI): 98.2-100] and a specificity of 56.8% (95% CI: 54.0-59.6). Automated methods of NI detection represent an alternative to traditional monitoring methods. Further study involving more ICUs should be performed before national recommendations can be established.

A comparison of methods to detect urinary tract infections using electronic data

BACKGROUND

The use of electronic medical records to identify common health care-associated infections (HAIs), including pneumonia, surgical site infections, bloodstream infections, and urinary tract infections (UTIs), has been proposed to help perform HAI surveillance and guide infection prevention efforts. Increased attention on HAIs has led to public health reporting requirements and a focus on quality improvement activities around HAIs. Traditional surveillance to detect HAIs and focus prevention efforts is labor intensive, and computer algorithms could be useful to screen electronic data and provide actionable information.

METHODS

Seven computer-based decision rules to identify UTIs were compared in a sample of 33,834 admissions to an urban academic health center. These decision rules included combinations of laboratory data, patient clinical data, and administrative data (for example, International Statistical Classification of Diseases and Related Health Problems, Ninth Revision [ICD-9] codes).

RESULTS

Of 33,834 hospital admissions, 3,870 UTIs were identified by at least one of the decision rules. The use of ICD-9 codes alone identified 2,614 UTIs. Laboratory-based definitions identified 2,773 infections, but when the presence of fever was included, only 1,125 UTIs were identified. The estimated sensitivity of ICD-9 codes was 55.6% (95% confidence interval [CI], 52.5%-58.5%) when compared with a culture- and symptom-based definition. Of the UTIs identified by ICD-9 codes, 167/1,125 (14.8%) also met two urine-culture decision rules.

DISCUSSION

Use of the example of UTI identification shows how different algorithms may be appropriate, depending on the goal of case identification. Electronic surveillance methods may be beneficial for mandatory reporting, process improvement, and economic analysis.

Developing information technology for infection prevention surveillance

The potential to automate at least part of the surveillance process for health care-associated infections was seen as soon as hospitals began to implement computer systems. Progress toward automated surveillance has been ongoing for the last several decades. But as more information becomes available electronically in the healthcare setting, the promise of electronic surveillance for healthcare-associated infections has become closer to reality. Although true fully automated surveillance is not here yet, significant progress is being made at a number of centers for electronic surveillance of central catheter-associated bloodstream infections, ventilator-associated pneumonia, and other healthcare-associated infections. We review the progress that has been made in this area and issues that need to be addressed as surveillance systems are implemented, as well as promising areas for future development.

Formulation of a model for automating infection surveillance: algorithmic detection of central-line associated bloodstream infection

OBJECTIVE

To formulate a model for translating manual infection control surveillance methods to automated, algorithmic approaches.

DESIGN

We propose a model for creating electronic surveillance algorithms by translating existing manual surveillance practices into automated electronic methods. Our model suggests that three dimensions of expert knowledge be consulted: clinical, surveillance, and informatics. Once collected, knowledge should be applied through a process of conceptualization, synthesis, programming, and testing.

RESULTS

We applied our framework to central vascular catheter associated bloodstream infection surveillance, a major healthcare performance outcome measure. We found that despite major barriers such as differences in availability of structured data, in types of databases used and in semantic representation of clinical terms, bloodstream infection detection algorithms could be deployed at four very diverse medical centers.

CONCLUSIONS

We present a framework that translates existing practice-manual infection detection-to an automated process for surveillance. Our experience details barriers and solutions discovered during development of electronic surveillance for central vascular catheter associated bloodstream infections at four hospitals in a variety of data environments. Moving electronic surveillance to the next level-availability at a majority of acute care hospitals nationwide-would be hastened by the incorporation of necessary data elements, vocabularies and standards into commercially available electronic health records.

Automated surveillance for central line-associated bloodstream infection in intensive care units

OBJECTIVE

To develop and evaluate computer algorithms with high negative predictive values that augment traditional surveillance for central line-associated bloodstream infection (CLABSI).

SETTING

Barnes-Jewish Hospital, a 1,250-bed tertiary care academic hospital in Saint Louis, Missouri.

METHODS

We evaluated all adult patients in intensive care units who had blood samples collected during the period from July 1, 2005, to June 30, 2006, that were positive for a recognized pathogen on culture. Each isolate recovered from culture was evaluated using the definitions for nosocomial CLABSI provided by the National Healthcare Safety Network of the Centers for Disease Control and Prevention. Using manual surveillance by infection prevention specialists as the gold standard, we assessed the ability of various combinations of dichotomous rules to determine whether an isolate was associated with a CLABSI. Sensitivity, specificity, and predictive values were calculated.

RESULTS

Infection prevention specialists identified 67 cases of CLABSI associated with 771 isolates recovered from blood samples. The algorithms excluded approximately 40%-62% of the isolates from consideration as possible causes of CLABSI. The simplest algorithm, with 2 dichotomous rules (ie, the collection of blood samples more than 48 hours after admission and the presence of a central venous catheter within 48 hours before collection of blood samples), had the highest negative predictive value (99.4%) and the lowest specificity (44.2%) for CLABSI. Augmentation of this algorithm with rules for common skin contaminants confirmed by another positive blood culture result yielded in a negative predictive value of 99.2% and a specificity of 68.0%.

CONCLUSIONS

An automated approach to surveillance for CLABSI that is characterized by a high negative predictive value can accurately identify and exclude positive culture results not representing CLABSI from further manual surveillance.