Uncovering Discrepancies in IV Vancomycin Infusion Records between Pump Logs and EHR Documentation

BACKGROUND

Infusion start time, completion time, and interruptions are the key data points needed in both area under the concentration-time curve (AUC)- and trough-based vancomycin therapeutic drug monitoring (TDM). However, little is known about the accuracy of documented times of drug infusions compared with automated recorded events in the infusion pump system. A traditional approach of direct observations of infusion practice is resource intensive and impractical to scale. We need a new methodology to leverage the infusion pump event logs to understand the prevalence of timestamp discrepancies as documented in the electronic health records (EHRs).

OBJECTIVES

We aimed to analyze timestamp discrepancies between EHR documentation (the information used for clinical decision making) and pump event logs (actual administration process) for vancomycin treatment as it may lead to suboptimal data used for therapeutic decisions.

METHODS

We used process mining to study the conformance between pump event logs and EHR data for a single hospital in the United States from July to December 2016. An algorithm was developed to link records belonging to the same infusions. We analyzed discrepancies in infusion start time, completion time, and interruptions.

RESULTS

Of the 1,858 infusions, 19.1% had infusion start time discrepancy more than ± 10 minutes. Of the 487 infusion interruptions, 2.5% lasted for more than 20 minutes before the infusion resumed. 24.2% (312 of 1,287) of 1-hour infusions and 32% (114 of 359) of 2-hour infusions had over 10-minute completion time discrepancy. We believe those discrepancies are inherent part of the current EHR documentation process commonly found in hospitals, not unique to the care facility under study.

CONCLUSION

We demonstrated pump event logs and EHR data can be utilized to study time discrepancies in infusion administration at scale. Such discrepancy should be further investigated at different hospitals to address the prevalence of the problem and improvement effort.

Detecting Unusual Intravenous Infusion Alerting Patterns with Machine Learning Algorithms

OBJECTIVE

To detect unusual infusion alerting patterns using machine learning (ML) algorithms as a first step to advance safer inpatient intravenous administration of high-alert medications.

MATERIALS AND METHODS

We used one year of detailed propofol infusion data from a hospital. Interpretable and clinically relevant variables were feature engineered, and data points were aggregated per calendar day. A univariate (maximum times-limit) moving range (mr) control chart was used to simulate clinicians' common approach to identifying unusual infusion alerting patterns. Three different unsupervised multivariate ML-based anomaly detection algorithms (Local Outlier Factor, Isolation Forest, and k-Nearest Neighbors) were used for the same purpose. Results from the control chart and ML algorithms were compared.

RESULTS

The propofol data had 3,300 infusion alerts, 92% of which were generated during the day shift and seven of which had a times-limit greater than 10. The mr-chart identified 15 alert pattern anomalies. Different thresholds were set to include the top 15 anomalies from each ML algorithm. A total of 31 unique ML anomalies were grouped and ranked by agreeability. All algorithms agreed on 10% of the anomalies, and at least two algorithms agreed on 36%. Each algorithm detected one specific anomaly that the mr-chart did not detect. The anomaly represented a day with 71 propofol alerts (half of which were overridden) generated at an average rate of 1.06 per infusion, whereas the moving alert rate for the week was 0.35 per infusion.

DISCUSSION

These findings show that ML-based algorithms are more robust than control charts in detecting unusual alerting patterns. However, we recommend using a combination of algorithms, as multiple algorithms serve a benchmarking function and allow researchers to focus on data points with the highest algorithm agreeability.

CONCLUSION

Unsupervised ML algorithms can assist clinicians in identifying unusual alert patterns as a first step toward achieving safer infusion practices.

Detecting Unusual Intravenous Infusion Alerting Patterns with Machine Learning Algorithms

OBJECTIVE

To detect unusual infusion alerting patterns using machine learning (ML) algorithms as a first step to advance safer inpatient intravenous administration of high-alert medications.

MATERIALS AND METHODS

We used one year of detailed propofol infusion data from a hospital. Interpretable and clinically relevant variables were feature engineered, and data points were aggregated per calendar day. A univariate (maximum times-limit) moving range (mr) control chart was used to simulate clinicians' common approach to identifying unusual infusion alerting patterns. Three different unsupervised multivariate ML-based anomaly detection algorithms (Local Outlier Factor, Isolation Forest, and k-Nearest Neighbors) were used for the same purpose. Results from the control chart and ML algorithms were compared.

RESULTS

The propofol data had 3,300 infusion alerts, 92% of which were generated during the day shift and seven of which had a times-limit greater than 10. The mr-chart identified 15 alert pattern anomalies. Different thresholds were set to include the top 15 anomalies from each ML algorithm. A total of 31 unique ML anomalies were grouped and ranked by agreeability. All algorithms agreed on 10% of the anomalies, and at least two algorithms agreed on 36%. Each algorithm detected one specific anomaly that the mr-chart did not detect. The anomaly represented a day with 71 propofol alerts (half of which were overridden) generated at an average rate of 1.06 per infusion, whereas the moving alert rate for the week was 0.35 per infusion.

DISCUSSION

These findings show that ML-based algorithms are more robust than control charts in detecting unusual alerting patterns. However, we recommend using a combination of algorithms, as multiple algorithms serve a benchmarking function and allow researchers to focus on data points with the highest algorithm agreeability.

CONCLUSION

Unsupervised ML algorithms can assist clinicians in identifying unusual alert patterns as a first step toward achieving safer infusion practices.

Quantifying the Impact of Infusion Alerts and Alarms on Nursing Workflows: A Retrospective Analysis

BACKGROUND

Smart infusion pumps affect workflows as they add alerts and alarms in an information-rich clinical environment where alarm fatigue is already a major concern. An analytic approach is needed to quantify the impact of these alerts and alarms on nursing workflows and patient safety.

OBJECTIVES

To analyze a detailed infusion dataset from a smart infusion pump system and identify contributing factors for infusion programming alerts, operational alarms, and alarm resolution times.

METHODS

We analyzed detailed infusion pump data across four hospitals in a health system for up to 1 year. The prevalence of alerts and alarms was grouped by infusion type and a selected list of 32 high-alert medications (HAMs). Logistic regression was used to explore the relationship between a set of risk factors and the occurrence of alerts and alarms. We used nonparametric tests to explore the relationship between alarm resolution times and a subset of predictor variables.

RESULTS

The study dataset included 745,641 unique infusions with a total of 3,231,300 infusion events. Overall, 28.7% of all unique infusions had at least one operational alarm, and 2.1% of all unique infusions had at least one programming alert. Alarms averaged two per infusion, whereas at least one alert happened in every 48 unique infusions. Eight percent of alarms took over 4 minutes to resolve. Intravenous fluid infusions had the highest rate of error-state occurrence. HAMs had 1.64 more odds for alerts than the rest of the infusions. On average, HAMs had a higher alert rate than maintenance fluids.

CONCLUSION

Infusion pump alerts and alarms impact clinical care, as alerts and alarms by design interrupt clinical workflow. Our study showcases how hospital system leadership teams can leverage infusion pump informatics to prioritize quality improvement and patient safety initiatives pertaining to infusion practices.

Investigating multiple sources of data for smart infusion pump and electronic health record interoperability

PURPOSE

Infusion pump data, which describe compliance to dose-error reduction software among other metrics, are retrievable from infusion pump vendor software, electronic health record (EHR) systems, and regional and national data repositories such as the Regenstrief National Center for Medical Device Informatics (REMEDI). Smart infusion pump and EHR interoperability has added to the granularity and complexity of data collected, and clinicians are challenged with efficiently comprehending and interpreting the data and reports available.

SUMMARY

Collaborative partnerships between the Indianapolis Coalition for Patient Safety and the Regenstrief Center for Healthcare Engineering allowed for clinicians, informaticists, researchers, and engineers to compare the information gained and strengths of using smart infusion pumps, EHR, and REMEDI to assess hospital medication safety in a setting of interoperability. Seven reporting capabilities were used to compare available reports, and 2 hypothetical scenarios were developed to highlight these processes. Infusion pump vendor-provided software and reports were found to provide the most usable information for detailed infusion reporting, while the EHR was strongly usable for interoperability compliance and REMEDI excelled in benchmarking capabilities.

CONCLUSION

While infusion analytics needs may differ across health systems, a better understanding of the strengths of infusion pump data and EHR data may help provide structure and direction in the infusion analytics process. Infusion data repositories such as REMEDI are useful tools to obtain information in a way not delivered by smart pump data.

Healthcare Professionals Risk Assessments for Alert Overrides in High-Risk IV Infusions Using Simulated Scenarios

This study aimed to use healthcare professionals' assessments to calculate expected risk of intravenous (IV) infusion harm for simulated high-risk medications that exceed soft limits and to investigate the impact of relevant risk factors. We designed 30 infusion scenarios for four high-risk medications, propofol, morphine, insulin, and heparin, infused in adult intensive care unit (AICU) and adult medical and surgical care unit (AMSU). A total of 20 pharmacists and 5 nurses provided their assessed expected risk of harm in each scenario. Descriptive statistics, analysis of variance with least square mean, and post hoc test were conducted to test the effects of field limit type, soft (SoftMax), and hard maximum drug limit types (HardMax), and care area-medication combination on risk of harm. The results showed that overdosing scenarios with continuous and bolus dose limit types were assessed with significantly higher risks than those of bolus dose rate type. An overdose infusion in AICU over a large SoftMax was assessed to be of higher risk than over a small one, but not in AMSU. For overdose infusions with three levels of drug amount, greater drug amount in AICU and AMSU was assessed to have higher risk, except insignificant risk difference between the infusions with higher and moderate drug amount in AMSU. This study obtained expected risk for simulated high-risk IV infusions and found that different field limit and SoftMax types can affect expected risk based on healthcare professionals' perspectives. The findings will be regarded as benchmarks for validating risk quantification models in future research.

Stakeholder perceptions of smart infusion pumps and drug library updates: A multisite, interdisciplinary study

PURPOSE

Results of a questionnaire-based study to evaluate smart infusion pump end users' perceptions and understanding of the drug library update process are reported.

METHODS

The Indianapolis Coalition for Patient Safety, Inc., in partnership with the Regenstrief Center for Healthcare Engineering, conducted a 33-item electronic, cross-sectional survey across 5 Indiana health systems from May through November 2017. Interdisciplinary participants identified for survey distribution included nurses, pharmacists, biomedical engineers, administrators, and medication safety officers. The survey assessed the following domains: patient safety, the drug library update process, knowledge of drug libraries and the update process, and end-user perceptions.

RESULTS

A total of 778 submitted surveys were included in the data analysis, with a large majority of responses (90.2%) provided by nurses. The use of drug libraries for ensuring patient safety was deemed extremely important or important by 88% of respondents, but 36% indicated that they were unsure of whether drug libraries are updated on a routine basis in their health system. Approximately two-thirds agreed that the current update process improves quality of care (65.0%) and patient safety (68.1%). Moreover, 53.3% agreed that the current drug library update process was effective. However, less than 10% responded correctly when asked about the steps required to update the drug library. Furthermore, only 18% correctly indicated that when a pump is on it may not necessarily contain the most up-to-date version of the drug library.

CONCLUSION

A survey of 5 health systems in Indianapolis identified several end-user knowledge gaps related to smart pump drug library updates. The results suggest that these gaps were most likely due to a combination of the 2-step update process and the fact that the current drug library version is not easy to find and/or user-friendly and it is unclear when an update is pending.

High-alert medication administration and intravenous smart pumps: A descriptive analysis of clinical practice

BACKGROUND

The Institute for Safe Medication Practices (ISMP) describes high alert medications (HAM) as medications that represent a heightened risk of patient harm when used in error. IV smart pumps with dose error reduction systems (DERS) were created to help address medication administration errors. Compliance with DERS provides a measure of how accurately a hospital uses smart pump technology to reduce IV medication error.

OBJECTIVE

The primary purpose of this research was to use the REMEDI dataset, an aggregate, multi-hospital database inclusive of smart pump analytics, to improve the current understanding of clinical practices for IV HAM administration.

METHODS

Descriptive analyses and analysis of variance (ANOVA) were used to test for differences in the mean DERS alert override rate, and mean DERS alert override to reprogram ratio between non-HAM and HAM overall, by hospital system, and by pump type.

RESULTS

High mean override rates for non-HAM (73.8%) and HAM (75.8%) and high override to reprogram ratios for both non-HAM (7.30) and HAM (9.92) were seen. No significant differences were found in override rates (p = 0.23) and override to reprogram ratios (p = 0.06) between non-HAM and HAM. By hospital system, significant variability in override rates and override to reprogram ratios were seen. By pump type, there were no significant differences in the mean override rates (Baxter: p = 0.09; BD p = 0.34; ICU Medical p = 0.18) and the mean override to reprogram ratios (Baxter p = 0.84; BD p = 0.03; ICU Medical p = 0.63) between non-HAM and HAM.

CONCLUSIONS

These findings indicate that the majority of alerts generated are bypassed by clinicians at the point of care, a symptom of alert fatigue. Given the potential for significant patient harm with HAM and the high DERS alert override rates that routinely occur during IV medication administration, this study provides further support for clinician-driven IV smart pump innovation to improve alert fatigue.

Tracking the Progress of Wireless Infusion Pump Drug Library Updates- A Data-Driven Analysis of Pump Update Delays

Modern smart infusion pumps are wirelessly connected to a network server for easy data communications. The two-way communication allows uploading of infusion data and downloading of drug library updates. We have discovered significant delays in library updates. This research aimed at studying the drug library update process of one vendor pump and the contributing factors of pump update delays. Our data included BD Alaris™ pump status and infusion reports of two hospital systems (92 and 80 days, respectively, in 2015). We analyzed drug library update progressions at the individual device and fleet levels. To complete a library update, a pump goes through two status transitions: from noncurrent to a new library pending, and from pending to current. On average it took five to nine days for 50% of a pump fleet to become current after a new drug library was disseminated. We confirmed factors that affect noncurrent-to-pending time to include time to first power-on and total power-on time. We also found that high pump utilization promotes shorter pending-to-current time. Two distinctive and important steps of a drug library update on Alaris™ pumps are pending a new library and completing the library installation. To avoid potential patient harm caused by infusion pumps without appropriate drug limits due to update delays, hospitals should monitor the progression of a drug library update on its pump fleet. Potential ways to improve drug library updates on a fleet of pumps include better technologies, improved pump user-interface design, and more staff training.

Big data in healthcare - the promises, challenges and opportunities from a research perspective: A case study with a model database

Recent advances in data collection during routine health care in the form of Electronic Health Records (EHR), medical device data (e.g., infusion pump informatics, physiological monitoring data, and insurance claims data, among others, as well as biological and experimental data, have created tremendous opportunities for biological discoveries for clinical application. However, even with all the advancement in technologies and their promises for discoveries, very few research findings have been translated to clinical knowledge, or more importantly, to clinical practice. In this paper, we identify and present the initial work addressing the relevant challenges in three broad categories: data, accessibility, and translation. These issues are discussed in the context of a widely used detailed database from an intensive care unit, Medical Information Mart for Intensive Care (MIMIC III) database.