An electronic catheter-associated urinary tract infection surveillance tool

Automated Identification of Postoperative Infections to Allow Prediction and Surveillance Based on Electronic Health Record Data: Scoping Review

BACKGROUND

Postoperative infections remain a crucial challenge in health care, resulting in high morbidity, mortality, and costs. Accurate identification and labeling of patients with postoperative bacterial infections is crucial for developing prediction models, validating biomarkers, and implementing surveillance systems in clinical practice.

OBJECTIVE

This scoping review aimed to explore methods for identifying patients with postoperative infections using electronic health record (EHR) data to go beyond the reference standard of manual chart review.

METHODS

We performed a systematic search strategy across PubMed, Embase, Web of Science (Core Collection), the Cochrane Library, and Emcare (Ovid), targeting studies addressing the prediction and fully automated surveillance (ie, without manual check) of diverse bacterial infections in the postoperative setting. For prediction modeling studies, we assessed the labeling methods used, categorizing them as either manual or automated. We evaluated the different types of EHR data needed for the surveillance and labeling of postoperative infections, as well as the performance of fully automated surveillance systems compared with manual chart review.

RESULTS

We identified 75 different methods and definitions used to identify patients with postoperative infections in studies published between 2003 and 2023. Manual labeling was the predominant method in prediction modeling research, 65% (49/75) of the identified methods use structured data, and 45% (34/75) use free text and clinical notes as one of their data sources. Fully automated surveillance systems should be used with caution because the reported positive predictive values are between 0.31 and 0.76.

CONCLUSIONS

There is currently no evidence to support fully automated labeling and identification of patients with infections based solely on structured EHR data. Future research should focus on defining uniform definitions, as well as prioritizing the development of more scalable, automated methods for infection detection using structured EHR data.

Development of a Periprocedure Trigger for Outpatient Interventional Radiology Procedures in the Veterans Health Administration

OBJECTIVES

Interventional radiology (IR) is the newest medical specialty. However, it lacks robust quality assurance metrics, including adverse event (AE) surveillance tools. Considering the high frequency of outpatient care provided by IR, automated electronic triggers offer a potential catalyst to support accurate retrospective AE detection.

METHODS

We programmed previously validated AE triggers (admission, emergency visit, or death up to 14 days after procedure) for elective, outpatient IR procedures performed in Veterans Health Administration surgical facilities between fiscal years 2017 and 2019. We then developed a text-based algorithm to detect AEs that explicitly occurred in the periprocedure time frame: before, during, and shortly after the IR procedure. Guided by the literature and clinical expertise, we generated clinical note keywords and text strings to flag cases with high potential for periprocedure AEs. Flagged cases underwent targeted chart review to measure criterion validity (i.e., the positive predictive value), to confirm AE occurrence, and to characterize the event.

RESULTS

Among 135,285 elective outpatient IR procedures, the periprocedure algorithm flagged 245 cases (0.18%); 138 of these had ≥1 AE, yielding a positive predictive value of 56% (95% confidence interval, 50%-62%). The previously developed triggers for admission, emergency visit, or death in 14 days flagged 119 of the 138 procedures with AEs (73%). Among the 43 AEs detected exclusively by the periprocedure trigger were allergic reactions, adverse drug events, ischemic events, bleeding events requiring blood transfusions, and cardiac arrest requiring cardiopulmonary resuscitation.

CONCLUSIONS

The periprocedure trigger performed well on IR outpatient procedures and offers a complement to other electronic triggers developed for outpatient AE surveillance.

Automating surveillance for healthcare-associated infections: Rationale and current realities (Part I/III)

Infection surveillance is one of the cornerstones of infection prevention and control. Measurement of process metrics and clinical outcomes, such as detection of healthcare-associated infections (HAIs), can be used to support continuous quality improvement. HAI metrics are reported as part of the CMS Hospital-Acquired Conditions Program, and they influence facility reputation and financial outcomes.

Development and validation of models for detection of postoperative infections using structured electronic health records data and machine learning

BACKGROUND

Postoperative infections constitute more than half of all postoperative complications. Surveillance of these complications is primarily done through manual chart review, which is time consuming, expensive, and typically only covers 10% to 15% of all operations. Automated surveillance would permit the timely evaluation of and reporting of all operations.

METHODS

The goal of this study was to develop and validate parsimonious, interpretable models for conducting surveillance of postoperative infections using structured electronic health records data. This was a retrospective study using 30,639 unique operations from 5 major hospitals between 2013 and 2019. Structured electronic health records data were linked to postoperative outcomes data from the American College of Surgeons National Surgical Quality Improvement Program. Predictors from the electronic health records included diagnoses, procedures, and medications. Infectious complications included surgical site infection, urinary tract infection, sepsis, and pneumonia within 30 days of surgery. The knockoff filter, a penalized regression technique that controls type I error, was applied for variable selection. Models were validated in a chronological held-out dataset.

RESULTS

Seven percent of patients experienced at least one type of postoperative infection. Models selected contained between 4 and 8 variables and achieved >0.91 area under the receiver operating characteristic curve, >81% specificity, >87% sensitivity, >99% negative predictive value, and 10% to 15% positive predictive value in a held-out test dataset.

CONCLUSION

Surveillance and reporting of postoperative infection rates can be implemented for all operations with high accuracy using electronic health records data and simple linear regression models.

Development and validation of a model for surveillance of postoperative bleeding complications using structured electronic health records data

BACKGROUND

Postoperative bleeding complications surveillance is done primarily through manual chart review. The purpose of this study was to develop and validate a detection model for postoperative bleeding complications using structured electronic health records data.

METHODS

Patients who underwent operations at 1 of 5 hospitals within our local health system between 2013 and 2019 and whose complications were reported by the American College of Surgeons National Surgical Quality Improvement Program were included. Electronic health records data were linked to American College of Surgeons National Surgical Quality Improvement Program data using personal health identifiers. Electronic health records predictors included diagnosis codes mapped to PheCodes, procedure names, and medications within 30 days after surgery. We defined bleeding events as the transfusion of red blood cell components within 30 days after surgery. The knockoff filter and the lasso were used to develop a model in a training set of operations from January 2013 to March 2017. Performance of each model was tested in a held-out data set of patients who underwent operations from March 2017 to October 2019.

RESULTS

A total of 30,639 patients were included; 1,112 patients (3.6%) had a bleeding event. Eight predictor variables were selected by the knockoff filter. When applied to the test set, specificity was 94%, sensitivity was 94%, area under the curve was 0.97, and accuracy was 93%. Calibration was consistent in lower predicted risk patients, whereas the model slightly overpredicted risk in high-risk patients.

CONCLUSION

We created a parsimonious, accurate model for identifying patients with bleeding complications. This model can be used to augment manual chart review for surveillance and reporting of perioperative bleeding complications, enabling inclusion of all surgeries in quality improvement efforts.

Diagnostic accuracy of electronic surveillance tool for catheter-associated urinary tract infections in tertiary care hospitals: A meta-analysis

BACKGROUND

Automated systems have been developed to reduce labor-intensive manual recordings during nosocomial infection surveillance. The diagnostic accuracies of these systems have differed in various settings.

METHODS

We designed this meta-analysis to evaluate the diagnostic accuracy of an electronic surveillance tool for catheter-associated urinary tract infections (CAUTIs) in tertiary care hospitals. We systematically searched databases such as Medline, Scopus, Cochrane library and Embase (from inception until November 2019) for relevant studies. We assessed the quality of trials using the diagnostic accuracy studies-2 tool, and performed a meta-analysis to obtain a pooled sensitivity and specificity for electronic surveillance. We included 6 studies with 16,492 patients in the analysis.

RESULTS

We found a pooled sensitivity of electronic diagnostic surveillance for CAUTIs of 97.5% (95% confidence interval [CI], 67.6-99.9%) and a pooled specificity of 92.6% (95% CI, 55.2-99.2%). The diagnostic odds ratio was 494 (95% CI, 89-2747). The positive likelihood ratio was 13.1 (95% CI, 1.63-105.8) and the negative likelihood ratio 0.02 (95% CI, 0.001-0.40). A bivariate box plot indicated the possibility of heterogeneity between the included studies.

CONCLUSION

Our review suggests that electronic surveillance is useful for diagnosing CAUTIs among hospitalized patients in tertiary care hospitals due to its high sensitivity and specificity.

The accuracy of fully automated algorithms for surveillance of healthcare-associated urinary tract infections in hospitalized patients

BACKGROUND

Surveillance for healthcare-associated infections such as healthcare-associated urinary tract infections (HA-UTI) is important for directing resources and evaluating interventions. However, traditional surveillance methods are resource-intensive and subject to bias.

AIM

To develop and validate a fully automated surveillance algorithm for HA-UTI using electronic health record (EHR) data.

METHODS

Five algorithms were developed using EHR data from 2979 admissions at Karolinska University Hospital from 2010 to 2011: (1) positive urine culture (UCx); (2) positive UCx + UTI codes (International Statistical Classification of Diseases and Related Health Problems, 10^th revision); (3) positive UCx + UTI-specific antibiotics; (4) positive UCx + fever and/or UTI symptoms; (5) algorithm 4 with negation for fever without UTI symptoms. Natural language processing (NLP) was used for processing free-text medical notes. The algorithms were validated in 1258 potential UTI episodes from January to March 2012 and results extrapolated to all UTI episodes within this period (N = 16,712). The reference standard for HA-UTIs was manual record review according to the European Centre for Disease Prevention and Control (and US Centers for Disease Control and Prevention) definitions by trained healthcare personnel.

FINDINGS

Of the 1258 UTI episodes, 163 fulfilled the ECDC HA-UTI definition and the algorithms classified 391, 150, 189, 194, and 153 UTI episodes, respectively, as HA-UTI. Algorithms 1, 2, and 3 had insufficient performances. Algorithm 4 achieved better performance and algorithm 5 performed best for surveillance purposes with sensitivity 0.667 (95% confidence interval: 0.594-0.733), specificity 0.997 (0.996-0.998), positive predictive value 0.719 (0.624-0.807) and negative predictive value 0.997 (0.996-0.997).

CONCLUSION

A fully automated surveillance algorithm based on NLP to find UTI symptoms in free-text had acceptable performance to detect HA-UTI compared to manual record review. Algorithms based on administrative and microbiology data only were not sufficient.

Electronically assisted surveillance systems of healthcare-associated infections: a systematic review

Background

Surveillance of healthcare-associated infections (HAI) is the basis of each infection control programme and, in case of acute care hospitals, should ideally include all hospital wards, medical specialties as well as all types of HAI. Traditional surveillance is labour intensive and electronically assisted surveillance systems (EASS) hold the promise to increase efficiency.

Objectives

To give insight in the performance characteristics of different approaches to EASS and the quality of the studies designed to evaluate them.

Methods

In this systematic review, online databases were searched and studies that compared an EASS with a traditional surveillance method were included. Two different indicators were extracted from each study, one regarding the quality of design (including reporting efficiency) and one based on the performance (e.g. specificity and sensitivity) of the EASS presented.

Results

A total of 78 studies were included. The majority of EASS (n = 72) consisted of an algorithm-based selection step followed by confirmatory assessment. The algorithms used different sets of variables. Only a minority (n = 7) of EASS were hospital-wide and designed to detect all types of HAI. Sensitivity of EASS was generally high (> 0.8), but specificity varied (0.37–1). Less than 20% (n = 14) of the studies presented data on the efficiency gains achieved.

Conclusions

Electronically assisted surveillance of HAI has yet to reach a mature stage and to be used routinely in healthcare settings. We recommend that future studies on the development and implementation of EASS of HAI focus on thorough validation, reproducibility, standardised datasets and detailed information on efficiency.

Identification of postoperative complications using electronic health record data and machine learning

BACKGROUND

Using the American College of Surgeons National Surgical Quality Improvement Program (NSQIP) complication status of patients who underwent an operation at the University of Colorado Hospital, we developed a machine learning algorithm for identifying patients with one or more complications using data from the electronic health record (EHR).

METHODS

We used an elastic-net model to estimate regression coefficients and carry out variable selection. International classification of disease codes (ICD-9), common procedural terminology (CPT) codes, medications, and CPT-specific complication event rate were included as predictors.

RESULTS

Of 6840 patients, 922 (13.5%) had at least one of the 18 complications tracked by NSQIP. The model achieved 88% specificity, 83% sensitivity, 97% negative predictive value, 52% positive predictive value, and an area under the curve of 0.93.

CONCLUSIONS

Using machine learning on EHR postoperative data linked to NSQIP outcomes data, a model with 163 predictors from the EHR identified complications well at our institution.

Accuracy of catheter-associated urinary tract infections reported to the National Healthcare Safety Network, January 2010 through July 2018

BACKGROUND

Surveillance of health care-associated, catheter-associated urinary tract infections (CAUTI) are the corner stone of infection prevention activity. The Centers for Disease Control and Prevention's National Healthcare Safety Network provides standard definitions for CAUTI surveillance, which have been updated periodically to increase objectivity, credibility, and reliability of urinary tract infection definitions. Several state health departments have validated CAUTI data that provided insights into accuracy of CAUTI reporting and adherence to CAUTI definition.

METHODS

Data accuracy measures included pooled mean sensitivity, specificity, positive predictive value, and negative predictive value. Total CAUTI error rate was computed as proportion of mismatches among total records. The impact of 2015 CAUTI definition changes were tested by comparing pooled accuracy estimates of validations prior to 2015 with post-2015.

RESULTS

At least 19 state health departments conducted CAUTI validations and indicated pooled mean sensitivity of 88.3%, specificity of 98.8%, positive predictive value of 93.6%, and negative predictive value of 97.6% of CAUTI reporting to the National Healthcare Safety Network. Among CAUTIs misclassified (121), 66% were underreported and 34% were overreported. CAUTI classification error rate declined significantly from 4.3% (pre-2015) to 2.4% (post-2015). Reasons for CAUTI misclassifications included: misapplication of CAUTI definition, misapplication of general health care-associated infection definitions, and clinical judgement over surveillance definition.

CONCLUSIONS

CAUTI underreporting is a major concern; validations provide transparency, education, and relationship building to improve reporting accuracy.

Electronic Surveillance For Catheter-Associated Urinary Tract Infection Using Natural Language Processing

Catheter-associated urinary tract infection (CAUTI) is a common and costly healthcare-associated infection, yet measuring it accurately is challenging and resource-intensive. Electronic surveillance promises to make this task more objective and efficient in an era of new financial and regulatory imperatives, but previous surveillance approaches have used a simplified version of the definition. We applied a complete definition, including subjective elements identified through natural language processing of clinical notes. Through examination of documentation practices, we defined a set of rules that identified positively and negatively asserted symptoms of CAUTI. Our algorithm was developed on a training set of 1421 catheterizedpatients and prospectively validated on 1567 catheterizedpatients. Compared to gold standard chart review, our tool had a sensitivity of 97.1%, specificity of 94.5% PPV of 66.7% and NPV of 99.6% for identifying CAUTI. We discuss sources of error and suggestions for more computable future definitions.

Internal and External Validation of a Computer-Assisted Surveillance System for Hospital-Acquired Infections in a 754-Bed General Hospital in the Netherlands

OBJECTIVE To evaluate a computer-assisted point-prevalence survey (CAPPS) for hospital-acquired infections (HAIs). DESIGN Validation cohort. SETTING A 754-bed teaching hospital in the Netherlands. METHODS For the internal validation of a CAPPS for HAIs, 2,526 patients were included. All patient records were retrospectively reviewed in depth by 2 infection control practitioners (ICPs) to determine which patients had suffered an HAI. Preventie van Ziekenhuisinfecties door Surveillance (PREZIES) criteria were used. Following this internal validation, 13 consecutive CAPPS were performed in a prospective study from January to March 2013 to determine weekly, monthly, and quarterly HAI point prevalence. Finally, a CAPPS was externally validated by PREZIES (Rijksinstituut voor Volksgezondheid en Milieu [RIVM], Bilthoven, Netherlands). In all evaluations, discrepancies were resolved by consensus. RESULTS In our series of CAPPS, 83% of the patients were automatically excluded from detailed review by the ICP. The sensitivity of the method was 91%. The time spent per hospital-wide CAPPS was ~3 hours. External validation showed a negative predictive value of 99.1% for CAPPS. CONCLUSIONS CAPPS proved to be a sensitive, accurate, and efficient method to determine serial weekly point-prevalence HAI rates in our hospital. Infect Control Hosp Epidemiol 2016;1-6.

A Web-Based, Hospital-Wide Health Care-Associated Bloodstream Infection Surveillance and Classification System: Development and Evaluation

BACKGROUND

Surveillance of health care-associated infections is an essential component of infection prevention programs, but conventional systems are labor intensive and performance dependent.

OBJECTIVE

To develop an automatic surveillance and classification system for health care-associated bloodstream infection (HABSI), and to evaluate its performance by comparing it with a conventional infection control personnel (ICP)-based surveillance system.

METHODS

We developed a Web-based system that was integrated into the medical information system of a 2200-bed teaching hospital in Taiwan. The system automatically detects and classifies HABSIs.

RESULTS

In this study, the number of computer-detected HABSIs correlated closely with the number of HABSIs detected by ICP by department (n=20; r=.999 P<.001) and by time (n=14; r=.941; P<.001). Compared with reference standards, this system performed excellently with regard to sensitivity (98.16%), specificity (99.96%), positive predictive value (95.81%), and negative predictive value (99.98%). The system enabled decreasing the delay in confirmation of HABSI cases, on average, by 29 days.

CONCLUSIONS

This system provides reliable and objective HABSI data for quality indicators, improving the delay caused by a conventional surveillance system.

Low correlation between self-report and medical record documentation of urinary tract infection symptoms

BACKGROUND

Correlations between symptom documentation in medical records and patient self-report (SR) vary depending on the condition studied. Patient symptoms are particularly important in urinary tract infection (UTI) diagnosis, and this correlation for UTI symptoms is currently unknown.

METHODS

This is a cross-sectional survey study in hospitalized patients with Escherichia coli bacteriuria. Patients were interviewed within 24 hours of diagnosis for the SR of UTI symptoms. We reviewed medical records for UTI symptoms documented by admitting or treating inpatient physicians (IPs), nurses (RNs), and emergency physicians (EPs). The level of agreement between groups was assessed using Cohen κ coefficient.

RESULTS

Out of 43 patients, 34 (79%) self-reported at least 1 of 6 primary symptoms. The most common self-reported symptoms were urinary frequency (53.5%); retention (41.9%); flank pain, suprapubic pain, and fatigue (37.2% each); and dysuria (30.2%). Correlation between SR and medical record documentation was slight to fair (κ, 0.06-0.4 between SR and IPs and 0.09-0.5 between SR and EDs). Positive agreement was highest for dysuria and frequency.

CONCLUSION

Correlation between self-reported UTI symptoms and health care providers' documentation was low to fair. Because medical records are a vital source of information for clinicians and researchers and symptom assessment and documentation are vital in distinguishing UTI from asymptomatic bacteriuria, efforts must be made to improve documentation.

Incidence rates of hospital-acquired urinary tract and bloodstream infections generated by automated compilation of electronically available healthcare data

BACKGROUND

Monitoring of hospital-acquired infection (HAI) by automated compilation of registry data may address the disadvantages of laborious, costly and potentially subjective and often random sampling of data by manual surveillance.

AIM

To evaluate a system for automated monitoring of hospital-acquired urinary tract (HA-UTI) and bloodstream infections (HA-BSI) and to report incidence rates over a five-year period in a Danish hospital trust.

METHODS

Based primarily on electronically available data relating to microbiology results and antibiotic prescriptions, the automated monitoring of HA-UTIs and HA-BSIs was validated against data from six previous point-prevalence surveys (PPS) from 2010 to 2013 and data from a manual assessment (HA-UTI only) of one department of internal medicine from January 2010. Incidence rates (infections per 1000 bed-days) from 2010 to 2014 were calculated.

FINDINGS

Compared with the PPSs, the automated monitoring showed a sensitivity of 88% in detecting UTI in general, 78% in detecting HA-UTI, and 100% in detecting BSI in general. The monthly incidence rates varied between 4.14 and 6.61 per 1000 bed-days for HA-UTI and between 0.09 and 1.25 per 1000 bed-days for HA-BSI.

CONCLUSION

Replacing PPSs with automated monitoring of HAIs may provide better and more objective data and constitute a promising foundation for individual patient risk analyses and epidemiological studies. Automated monitoring may be universally applicable in hospitals with electronic databases comprising microbiological findings, admission data, and antibiotic prescriptions.

Data elements and validation methods used for electronic surveillance of health care-associated infections: a systematic review

BACKGROUND

We describe the primary data sources, data elements, and validation methods currently used in electronic surveillance systems (ESS) for identification and surveillance of health care-associated infections (HAIs), and compares these data elements and validation methods with recommended standards.

METHODS

Using Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, a PubMed and manual search was conducted to identify research articles describing ESS for identification and surveillance of HAIs published January 1, 2009-August 31, 2014. Selected articles were evaluated to determine what data elements and validation methods were included.

RESULTS

Among the 509 articles identified in the original literature search, 30 met the inclusion criteria. Whereas the majority of studies (83%) used recommended data sources and validated the numerator (80%), only 10% of studies performed external and internal validation. In addition, there was variation in the ESS data formats used.

CONCLUSIONS

Our findings suggest that the majority of ESS for HAI surveillance use standard definitions, but the lack of widespread internal data, denominator, and external validation in these systems reduces the reliability of their findings. Additionally, advanced programming skills are required to create, implement, and maintain these systems and to reduce the variability in data formats.

An electronic surveillance tool for catheter-associated urinary tract infection in intensive care units

BACKGROUND

Traditional methods of surveillance of catheter-associated urinary tract infections (CAUTIs) are error-prone and resource-intensive. To resolve these issues, we developed a highly sensitive electronic surveillance tool.

OBJECTIVE

To develop an electronic surveillance tool for CAUTIs and assess its performance.

METHODS

The study was conducted at a 947-bed tertiary care center. Patients included adults aged ≥18 years admitted to an intensive care unit between January 10 and June 30, 2012, with an indwelling urinary catheter during their admission. We identified CAUTIs using 4 methods: traditional surveillance (TS) (ie, manual chart review by ICPs), an electronic surveillance (ES) tool, augmented electronic surveillance (AES) (ie, ES with chart review on a subset of cases), and reference standard (RS) (ie, a subset of CAUTIs originally ascertained by TS or ES, confirmed by review). We assessed performance characteristics to RS for reviewed cases.

RESULTS

We identified 417 candidate CAUTIs in 308 patients; 175 (42.0%) of these candidate CAUTIs were selected for review, yielding 32 confirmed CAUTIs in 22 patients (RS). Compared with RS, the sensitivities of TS, ES, and AES were 43.8% (95% confidence interval [CI], 26.4%-62.3%), 100.0% (95% CI, 89.1%-100.0%), and 100.0% (95% CI, 89.1%-100.0%). Specificities were 82.5% (95% CI, 75.3%-88.4%), 2.8% (95% CI, 0.8%-7.0%), and 100.0% (95% CI, 97.5%-100.0%).

CONCLUSIONS

Electronic CAUTI surveillance offers a streamlined approach to improve reliability and resource burden of surveillance.

Natural Language Processing for Real-Time Catheter-Associated Urinary Tract Infection Surveillance: Results of a Pilot Implementation Trial

BACKGROUND

Incidence of catheter-associated urinary tract infection (CAUTI) is a quality benchmark. To streamline conventional detection methods, an electronic surveillance system augmented with natural language processing (NLP), which gathers data recorded in clinical notes without manual review, was implemented for real-time surveillance.

OBJECTIVE

To assess the utility of this algorithm for identifying indwelling urinary catheter days and CAUTI.

SETTING

Large, urban tertiary care Veterans Affairs hospital.

METHODS

All patients admitted to the acute care units and the intensive care unit from March 1, 2013, through November 30, 2013, were included. Standard surveillance, which includes electronic and manual data extraction, was compared with the NLP-augmented algorithm.

RESULTS

The NLP-augmented algorithm identified 27% more indwelling urinary catheter days in the acute care units and 28% fewer indwelling urinary catheter days in the intensive care unit. The algorithm flagged 24 CAUTI versus 20 CAUTI by standard surveillance methods; the CAUTI identified were overlapping but not the same. The overall positive predictive value was 54.2%, and overall sensitivity was 65% (90.9% in the acute care units but 33% in the intensive care unit). Dissimilarities in the operating characteristics of the algorithm between types of unit were due to differences in documentation practice. Development and implementation of the algorithm required substantial upfront effort of clinicians and programmers to determine current language patterns.

CONCLUSIONS

The NLP algorithm was most useful for identifying simple clinical variables. Algorithm operating characteristics were specific to local documentation practices. The algorithm did not perform as well as standard surveillance methods.

Accuracy of Electronic Surveillance of Catheter-Associated Urinary Tract Infection at an Academic Medical Center

Objective.

To develop and validate a methodology for electronic surveillance of catheter-associated urinary tract infections (CAUTIs).

Design.

Diagnostic accuracy study.

Setting.

A 425-bed university hospital.

Subjects.

A total of 1,695 unique inpatient encounters from November 2009 through November 2010 with a high clinical suspicion of CAUTI.

Methods.

An algorithm was developed to identify incident CAUTIs from electronic health records (EHRs) on the basis of the Centers for Disease Control and Prevention (CDC) surveillance definition. CAUTIs identified by electronic surveillance were compared with the reference standard of manual surveillance by infection preventionists. To determine diagnostic accuracy, we created 2 × 2 tables, one unadjusted and one adjusted for misclassification using chart review and case adjudication. Unadjusted and adjusted test statistics (percent agreement, sensitivity, specificity, positive predictive value [PPV], negative predictive value [NPV], and κ) were calculated.

Results.

Electronic surveillance identified 64 CAUTIs compared with manual surveillance, which identified 19 CAUTIs for 97% agreement, 79% sensitivity, 97% sensitivity, 23% PPV, 100% NPV, and κ of .33. Compared with the reference standard adjusted for misclassification, which identified 55 CAUTIs, electronic surveillance had 98% agreement, 80% sensitivity, 99% specificity, 69% PPV, 99% NPV, and κ of .71.

Conclusion.

The electronic surveillance methodology had a high NPV and a low PPV compared with the reference standard, indicating a role of the electronic algorithm in screening data sets to exclude cases. However, the PPV markedly improved compared with the reference standard adjusted for misclassification, suggesting a future role in surveillance with improvements in EHRs.

Infect Control Hosp Epidemiol2014;35(6):685–691

Factors influencing the self-perceived practice levels of professional standard competency among infection control nurses in Korea

BACKGROUND

This study investigated the self-perceived infection control (IC) knowledge and practice levels of professional standards competency (PSC) among Korean infection control nurses (ICNs) to identify factors that may influence PSC.

METHODS

Using a self-reporting questionnaire method, we collected data from a total of 104 ICNs.

RESULTS

The average self-perceived IC knowledge level was 3.1 ± 0.8, with hand hygiene scoring the highest at 3.7 ± 0.8. The total proportion of responders who did not meet the expected standard in 4 future-oriented domains was 51.7%. Of the 4 domains, technology had the highest number of respondents meeting the desired standard (57%). There were significant differences in self-perceived levels of PSC in relation to ICN specialist certification and continuing education (eg, extra coursework, conference attendance) in the field. Self-perceived practice levels of PSC also were significantly correlated with age, years of total clinical experience, years of ICN experience, hospital bed count, and IC knowledge. Predictors of self-perceived practice levels of PSC were knowledge and years of ICN experience.

CONCLUSION

Educational programs are needed to promote knowledge and competency, the lack of which was recognized by the ICNs. Also, various efforts are needed to prevent turnover of ICNs with a high level of competency.

Accuracy of electronic surveillance of catheter-associated urinary tract infection at an academic medical center

OBJECTIVE

To develop and validate a methodology for electronic surveillance of catheter-associated urinary tract infections (CAUTIs).

DESIGN

Diagnostic accuracy study.

SETTING

A 425-bed university hospital.

SUBJECTS

A total of 1,695 unique inpatient encounters from November 2009 through November 2010 with a high clinical suspicion of CAUTI.

METHODS

An algorithm was developed to identify incident CAUTIs from electronic health records (EHRs) on the basis of the Centers for Disease Control and Prevention (CDC) surveillance definition. CAUTIs identified by electronic surveillance were compared with the reference standard of manual surveillance by infection preventionists. To determine diagnostic accuracy, we created 2 × 2 tables, one unadjusted and one adjusted for misclassification using chart review and case adjudication. Unadjusted and adjusted test statistics (percent agreement, sensitivity, specificity, positive predictive value [PPV], negative predictive value [NPV], and κ) were calculated.

RESULTS

Electronic surveillance identified 64 CAUTIs compared with manual surveillance, which identified 19 CAUTIs for 97% agreement, 79% sensitivity, 97% sensitivity, 23% PPV, 100% NPV, and κ of .33. Compared with the reference standard adjusted for misclassification, which identified 55 CAUTIs, electronic surveillance had 98% agreement, 80% sensitivity, 99% specificity, 69% PPV, 99% NPV, and κ of .71.

CONCLUSION

The electronic surveillance methodology had a high NPV and a low PPV compared with the reference standard, indicating a role of the electronic algorithm in screening data sets to exclude cases. However, the PPV markedly improved compared with the reference standard adjusted for misclassification, suggesting a future role in surveillance with improvements in EHRs.

Data use and effectiveness in electronic surveillance of healthcare associated infections in the 21st century: a systematic review

OBJECTIVE

As more electronic health records have become available during the last decade, we aimed to uncover recent trends in use of electronically available patient data by electronic surveillance systems for healthcare associated infections (HAIs) and identify consequences for system effectiveness.

METHODS

A systematic review of published literature evaluating electronic HAI surveillance systems was performed. The PubMed service was used to retrieve publications between January 2001 and December 2011. Studies were included in the review if they accurately described what electronic data were used and if system effectiveness was evaluated using sensitivity, specificity, positive predictive value, or negative predictive value. Trends were identified by analyzing changes in the number and types of electronic data sources used.

RESULTS

26 publications comprising discussions on 27 electronic systems met the eligibility criteria. Trend analysis showed that systems use an increasing number of data sources which are either medico-administrative or clinical and laboratory-based data. Trends on the use of individual types of electronic data confirmed the paramount role of microbiology data in HAI detection, but also showed increased use of biochemistry and pharmacy data, and the limited adoption of clinical data and physician narratives. System effectiveness assessments indicate that the use of heterogeneous data sources results in higher system sensitivity at the expense of specificity.

CONCLUSIONS

Driven by the increased availability of electronic patient data, electronic HAI surveillance systems use more data, making systems more sensitive yet less specific, but also allow systems to be tailored to the needs of healthcare institutes' surveillance programs.

Quality gaps in documenting urinary catheter use and infectious outcomes

OBJECTIVE

To describe the frequency of use of all types of urinary catheters, including but not limited to indwelling catheters, as well as positive cultures associated with the various types. We also determined the accuracy of catheter-days reporting at our institution.

DESIGN

Prospective, observational trial based on patient-level review of the electronic medical record. Chart review was compared with standard methods of catheter surveillance and reporting by infection control personnel.

SETTING

Ten internal medicine and 5 long-term care wards in 2 tertiary care Veterans Affairs hospitals in Texas from July 2010 through June 2011.

PARTICIPANTS

The study included 7,866 inpatients.

METHODS

Measurements included patient bed-days; days of use of indwelling, external, suprapubic, and intermittent urinary catheters; number of urine cultures obtained and culture results; and infection control reports of indwelling catheter-days.

RESULTS

We observed 7,866 inpatients with 128,267 bed-days on acute medicine and extended care wards during the study. A urinary catheter was used on 36.9% of the total bed-days observed. Acute medicine wards collected more urine cultures per 1,000 bed-days than did the extended care wards (75.9 and 10.4 cultures per 1,000 bed-days, respectively; P<.001). Catheter-days were divided among indwelling-catheter-days (47.8%), external-catheter-days (48.4%), and other (intermittent- and suprapubic-catheter-days, 3.8%). External catheters contributed to 376 (37.3%) of the 1,009 catheter-associated positive urine cultures. Urinary-catheter-days reported to the infection control department missed 20.1% of the actual days of indwelling catheter use, whereas 12.0% of their reported catheter-days were false.

CONCLUSIONS

Urinary catheter use was extremely common. External catheters accounted for a large portion of catheter-associated bacteriuria, and standard practices for tracking urinary-catheter-days were unreliable.

TRIAL REGISTRATION

ClinicalTrials.gov identifier: NCT01052545.

Diagnostic performance of electronic syndromic surveillance systems in acute care: a systematic review

CONTEXT

Healthcare Electronic Syndromic Surveillance (ESS) is the systematic collection, analysis and interpretation of ongoing clinical data with subsequent dissemination of results, which aid clinical decision-making.

OBJECTIVE

To evaluate, classify and analyze the diagnostic performance, strengths and limitations of existing acute care ESS systems.

DATA SOURCES

All available to us studies in Ovid MEDLINE, Ovid EMBASE, CINAHL and Scopus databases, from as early as January 1972 through the first week of September 2012.

STUDY SELECTION

Prospective and retrospective trials, examining the diagnostic performance of inpatient ESS and providing objective diagnostic data including sensitivity, specificity, positive and negative predictive values.

DATA EXTRACTION

Two independent reviewers extracted diagnostic performance data on ESS systems, including clinical area, number of decision points, sensitivity and specificity. Positive and negative likelihood ratios were calculated for each healthcare ESS system. A likelihood matrix summarizing the various ESS systems performance was created.

RESULTS

The described search strategy yielded 1639 articles. Of these, 1497 were excluded on abstract information. After full text review, abstraction and arbitration with a third reviewer, 33 studies met inclusion criteria, reporting 102,611 ESS decision points. The yielded I2 was high (98.8%), precluding meta-analysis. Performance was variable, with sensitivities ranging from 21% -100% and specificities ranging from 5%-100%.

CONCLUSIONS

There is significant heterogeneity in the diagnostic performance of the available ESS implements in acute care, stemming from the wide spectrum of different clinical entities and ESS systems. Based on the results, we introduce a conceptual framework using a likelihood ratio matrix for evaluation and meaningful application of future, frontline clinical decision support systems.

Advances in electronic surveillance for healthcare-associated infections in the 21st Century: a systematic review

BACKGROUND

Traditional methodologies for healthcare-associated infection (HCAI) surveillance can be resource intensive and time consuming. As a consequence, surveillance is often limited to specific organisms or conditions. Various electronic databases exist within the healthcare setting and may be utilized to perform HCAI surveillance.

AIM

To assess the utility of electronic surveillance systems for monitoring and detecting HCAI.

METHODS

A systematic review of published literature on surveillance of HCAI was performed. Databases were searched for studies published between January 2000 and December 2011. Search terms were divided into infection, surveillance and data management terms, and combined using Boolean operators. Studies were included for review if they demonstrated or proposed the use of electronic systems for HCAI surveillance.

FINDINGS

In total, 44 studies met the inclusion criteria. For the majority of studies, emphasis was on the linkage of electronic databases to provide automated methods for monitoring infections in specific clinical settings. Twenty-one studies assessed the performance of their method with traditional surveillance methodologies or a manual reference method. Where sensitivity and specificity were calculated, these varied depending on the organism or condition being surveyed and the data sources employed.

CONCLUSIONS

The implementation of electronic surveillance was found to be feasible in many settings, with several systems fully integrated into hospital information systems and routine surveillance practices. The results of this review suggest that electronic surveillance systems should be developed to maximize the efficacy of abundant electronic data sources existing within hospitals.

A comparison of 3 metrics to identify health care-associated infections

BACKGROUND

The best approach to measurement of health care-associated infection rates is controversial.

METHODS

We compared 3 metrics to identify catheter-associated bloodstream infection (CA-BSI), catheter-associated urinary tract infection (CA-UTI), and ventilator-associated pneumonia (VAP) in 8 intensive care units during 2009. We evaluated traditional surveillance using National Healthcare Safety Network methodology, data mining with MedMined Data Mining Surveillance (CareFusion Corporation, San Diego, CA), and administrative coding with ICD-9-CM.

RESULTS

A total of 65 CA-BSI, 28 CA-UTI, and 48 VAP was identified. Traditional surveillance detected 58 CA-BSI and no false positives; data mining identified 51 cases but 51 false positives; administrative coding documented 6 cases and 6 false positives. Traditional surveillance detected 27 CA-UTI and no false positives; data mining identified 17 cases but 19 false positives; administrative coding documented 3 cases and 1 false-positive. Traditional surveillance detected 41 VAP and no false positives; data mining identified 26 cases but also 79 false positives; administrative coding found 17 cases and 13 false positives. Overall sensitivities were as follows: traditional surveillance, 0.84; data mining, 0.67; administrative coding, 0.18. Positive predictive values were as follows: traditional surveillance, 1.0; data mining, 0.39; administrative coding, 0.57.

CONCLUSION

Traditional surveillance proved superior in terms of sensitivity, positive predictive value, and rate estimation.

Competency in infection prevention: a conceptual approach to guide current and future practice

Professional competency has traditionally been divided into 2 essential components: knowledge and skill. More recent definitions have recommended additional components such as communication, values, reasoning, and teamwork. A standard, widely accepted, comprehensive definition remains an elusive goal. For infection preventionists (IPs), the requisite elements of competence are most often embedded in the IP position description, which may or may not reference national standards or guidelines. For this reason, there is widespread variation among these elements and the criteria they include. As the demand for IP expertise continues to rapidly expand, the Association for Professionals in Infection Control and Epidemiology, Inc, made a strategic commitment to develop a conceptual model of IP competency that could be applicable in all practice settings. The model was designed to be used in combination with organizational training and evaluation tools already in place. Ideally, the Association for Professionals in Infection Control and Epidemiology, Inc, model will complement similar competency efforts undertaken in non-US countries and/or international organizations. This conceptual model not only describes successful IP practice as it is today but is also meant to be forward thinking by emphasizing those areas that will be especially critical in the next 3 to 5 years. The paper also references a skill assessment resource developed by Community and Hospital Infection Control Association (CHICA)-Canada and a competency model developed by the Infection Prevention Society (IPS), which offer additional support of infection prevention as a global patient safety mission.

The dilemma of assessment bias in infection control research

Infection control studies often rely on infection endpoints to determine whether interventions are effective. However, many infection outcomes, including those defined by standardized surveillance criteria, involve some subjective judgment for determination. Studies that use unblinded ascertainment of subjective infection endpoints are at risk of assessment bias. Unfortunately, infection control studies have not routinely accounted for assessment bias. To ensure validity, infection control studies should incorporate study design elements to control assessment bias, such as blinded assessment or use of objective outcome measures.