Assessing data quality in manual entry of ventilator settings

PVP1-The People's Ventilator Project: A fully open, low-cost, pressure-controlled ventilator research platform compatible with adult and pediatric uses

Mechanical ventilators are safety-critical devices that help patients breathe, commonly found in hospital intensive care units (ICUs)-yet, the high costs and proprietary nature of commercial ventilators inhibit their use as an educational and research platform. We present a fully open ventilator device-The People's Ventilator: PVP1-with complete hardware and software documentation including detailed build instructions and a DIY cost of $1,700 USD. We validate PVP1 against both key performance criteria specified in the U.S. Food and Drug Administration's Emergency Use Authorization for Ventilators, and in a pediatric context against a state-of-the-art commercial ventilator. Notably, PVP1 performs well over a wide range of test conditions and performance stability is demonstrated for a minimum of 75,000 breath cycles over three days with an adult mechanical test lung. As an open project, PVP1 can enable future educational, academic, and clinical developments in the ventilator space.

Comparing automated vs. manual data collection for COVID-specific medications from electronic health records

INTRODUCTION

Data extraction from electronic health record (EHR) systems occurs through manual abstraction, automated extraction, or a combination of both. While each method has its strengths and weaknesses, both are necessary for retrospective observational research as well as sudden clinical events, like the COVID-19 pandemic. Assessing the strengths, weaknesses, and potentials of these methods is important to continue to understand optimal approaches to extracting clinical data. We set out to assess automated and manual techniques for collecting medication use data in patients with COVID-19 to inform future observational studies that extract data from the electronic health record (EHR).

MATERIALS AND METHODS

For 4,123 COVID-positive patients hospitalized and/or seen in the emergency department at an academic medical center between 03/03/2020 and 05/15/2020, we compared medication use data of 25 medications or drug classes collected through manual abstraction and automated extraction from the EHR. Quantitatively, we assessed concordance using Cohen's kappa to measure interrater reliability, and qualitatively, we audited observed discrepancies to determine causes of inconsistencies.

RESULTS

For the 16 inpatient medications, 11 (69%) demonstrated moderate or better agreement; 7 of those demonstrated strong or almost perfect agreement. For 9 outpatient medications, 3 (33%) demonstrated moderate agreement, but none achieved strong or almost perfect agreement. We audited 12% of all discrepancies (716/5,790) and, in those audited, observed three principal categories of error: human error in manual abstraction (26%), errors in the extract-transform-load (ETL) or mapping of the automated extraction (41%), and abstraction-query mismatch (33%).

CONCLUSION

Our findings suggest many inpatient medications can be collected reliably through automated extraction, especially when abstraction instructions are designed with data architecture in mind. We discuss quality issues, concerns, and improvements for institutions to consider when crafting an approach. During crises, institutions must decide how to allocate limited resources. We show that automated extraction of medications is feasible and make recommendations on how to improve future iterations.

Quantifying Mobility in the ICU: Comparison of Electronic Health Record Documentation and Accelerometer-Based Sensors to Clinician-Annotated Video

Supplemental Digital Content is available in the text.

To compare the accuracy of electronic health record clinician documentation and accelerometer-based sensors with a gold standard dataset derived from clinician-annotated video to quantify early mobility activities in adult ICU patients.

The dawn of physiological closed-loop ventilation-a review

The level of automation in mechanical ventilation has been steadily increasing over the last few decades. There has recently been renewed interest in physiological closed-loop control of ventilation. The development of these systems has followed a similar path to that of manual clinical ventilation, starting with ensuring optimal gas exchange and shifting to the prevention of ventilator-induced lung injury. Systems currently aim to encompass both aspects, and early commercial systems are appearing. These developments remain unknown to many clinicians and, hence, limit their adoption into the clinical environment. This review shows the evolution of the physiological closed-loop control of mechanical ventilation.

Representativeness comparisons of nurse and computer charting of heart rate across nursing-intensity protocols

Clinical teams in acute inpatient settings can greatly benefit from automated charting technologies that continuously monitor patient vital status. NewYork-Presbyterian has designed and developed a real-time patient monitoring system that integrates vital signs sensors, networking, and electronic health records, to allow for automatic charting of patient status. We evaluate the representativeness (a combination of agreement, safety and timing) of a core vital sign across nursing intensity care protocols for preliminary feasibility assessment. Our findings suggest an automated way of summarizing heart rate provides representation of true heart rate status and can facilitate alternatives approaches to burdensome manual nurse charting of physiological parameters.

Data analytics for continuous renal replacement therapy: historical limitations and recent technology advances

PURPOSE

Dialysis is a highly quantitative therapy involving large volumes of both clinical and technical data. While automated data collection has been implemented for chronic dialysis, this has not been done for acute kidney injury patients treated with continuous renal replacement therapy (CRRT).

METHODS

After a brief review of the fundamental aspects of electronic medical records (EMRs), a new tool designed to provide clinicians with individualized CRRT treatment data is analyzed, with emphasis on its quality assurance capabilities.

RESULTS

The first platform addressing the problem of data collection and management with current CRRT machines (Sharesource system; Baxter Healthcare) is described. The system provides connectivity for the Prismaflex CRRT machine and enables both EMR connectivity and therapy analytics with 2 basic components: the connect module and the report module.

CONCLUSIONS

The enormous amount of data in CRRT should be collected and analyzed to enable adequate clinical decisions. Current CRRT technology presents significant limitations with consequent lack of rigorous analysis of technical data and relevant feedback. From a quality assurance perspective, these limitations preclude any systematic assessment of prescription and delivery trends that may be adversely affecting clinical outcomes. A detailed assessment of current practice limitations is provided together with several possible ways to address such limitations by a new technical tool.

Clinical Information Systems - From Yesterday to Tomorrow

OBJECTIVES

To review the history of clinical information systems over the past twenty-five years and project anticipated changes to those systems over the next twenty-five years.

METHODS

Over 250 Medline references about clinical information systems, quality of patient care, and patient safety were reviewed. Books, Web resources, and the author's personal experience with developing the HELP system were also used.

RESULTS

There have been dramatic improvements in the use and acceptance of clinical computing systems and Electronic Health Records (EHRs), especially in the United States. Although there are still challenges with the implementation of such systems, the rate of progress has been remarkable. Over the next twenty-five years, there will remain many important opportunities and challenges. These opportunities include understanding complex clinical computing issues that must be studied, understood and optimized. Dramatic improvements in quality of care and patient safety must be anticipated as a result of the use of clinical information systems. These improvements will result from a closer involvement of clinical informaticians in the optimization of patient care processes.

CONCLUSIONS

Clinical information systems and computerized clinical decision support have made contributions to medicine in the past. Therefore, by using better medical knowledge, optimized clinical information systems, and computerized clinical decision, we will enable dramatic improvements in both the quality and safety of patient care in the next twenty-five years.

Integrating Real Time Data to Improve Outcomes in Acute Kidney Injury

Critically ill patients with acute kidney injury requiring renal replacement therapy have a poor prognosis. Despite well-known factors, which contribute to outcomes, including dose delivery, patients frequently miss the target dose and volume removal. One major barrier to effective care of these patients is the traditional dissociation of dialysis device data from other clinical information systems, notably the electronic health record (EHR). This lack of integration and the resulting manual documentation leads to errors and biases in documentation and missed opportunities to intervene in a timely fashion. This review summarizes the technological advancements facilitating direct connection of dialysis devices to EHRs. This connection facilitates automated data capture of many variables - including delivered dose, ultrafiltration rate and pressure measurements - which in turn can be leveraged for data mining, quality improvement and real-time targeted therapy adjustments. These interventions hold the promise to significantly improve outcomes for this patient population.

Challenges Associated With Using Large Data Sets for Quality Assessment and Research in Clinical Settings

The rapidly expanding use of electronic records in health-care settings is generating unprecedented quantities of data available for clinical, epidemiological, and cost-effectiveness research. Several challenges are associated with using these data for clinical research, including issues surrounding access and information security, poor data quality, inconsistency of data within and across institutions, and a paucity of staff with expertise to manage and manipulate large clinical data sets. In this article, we describe our experience with assembling a data-mart and conducting clinical research using electronic data from four facilities within a single hospital network in New York City. We culled data from several electronic sources, including the institution's admission-discharge-transfer system, cost accounting system, electronic health record, clinical data warehouse, and departmental records. The final data-mart contained information for more than 760,000 discharges occurring from 2006 through 2012. Using categories identified by the National Institutes of Health Big Data to Knowledge initiative as a framework, we outlined challenges encountered during the development and use of a domain-specific data-mart and recommend approaches to overcome these challenges.

Sharing Vital Signs between mobile phone applications

We propose a communication library, ShareVitalSigns, for the standardized exchange of vital sign information between health applications running on mobile platforms. The library allows an application to request one or multiple vital signs from independent measurement applications on the Android OS. Compatible measurement applications are automatically detected and can be launched from within the requesting application, simplifying the work flow for the user and reducing typing errors. Data is shared between applications using intents, a passive data structure available on Android OS. The library is accompanied by a test application which serves as a demonstrator. The secure exchange of vital sign information using a standardized library like ShareVitalSigns will facilitate the integration of measurement applications into diagnostic and other high level health monitoring applications and reduce errors due to manual entry of information.

Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research

OBJECTIVE

To review the methods and dimensions of data quality assessment in the context of electronic health record (EHR) data reuse for research.

MATERIALS AND METHODS

A review of the clinical research literature discussing data quality assessment methodology for EHR data was performed. Using an iterative process, the aspects of data quality being measured were abstracted and categorized, as well as the methods of assessment used.

RESULTS

Five dimensions of data quality were identified, which are completeness, correctness, concordance, plausibility, and currency, and seven broad categories of data quality assessment methods: comparison with gold standards, data element agreement, data source agreement, distribution comparison, validity checks, log review, and element presence.

DISCUSSION

Examination of the methods by which clinical researchers have investigated the quality and suitability of EHR data for research shows that there are fundamental features of data quality, which may be difficult to measure, as well as proxy dimensions. Researchers interested in the reuse of EHR data for clinical research are recommended to consider the adoption of a consistent taxonomy of EHR data quality, to remain aware of the task-dependence of data quality, to integrate work on data quality assessment from other fields, and to adopt systematic, empirically driven, statistically based methods of data quality assessment.

CONCLUSION

There is currently little consistency or potential generalizability in the methods used to assess EHR data quality. If the reuse of EHR data for clinical research is to become accepted, researchers should adopt validated, systematic methods of EHR data quality assessment.

The future state of clinical data capture and documentation: a report from AMIA's 2011 Policy Meeting

Much of what is currently documented in the electronic health record is in response toincreasingly complex and prescriptive medicolegal, reimbursement, and regulatory requirements. These requirements often result in redundant data capture and cumbersome documentation processes. AMIA's 2011 Health Policy Meeting examined key issues in this arena and envisioned changes to help move toward an ideal future state of clinical data capture and documentation. The consensus of the meeting was that, in the move to a technology-enabled healthcare environment, the main purpose of documentation should be to support patient care and improved outcomes for individuals and populations and that documentation for other purposes should be generated as a byproduct of care delivery. This paper summarizes meeting deliberations, and highlights policy recommendations and research priorities. The authors recommend development of a national strategy to review and amend public policies to better support technology-enabled data capture and documentation practices.

Neonatal Informatics: Optimizing Clinical Data Entry and Display

Displaying the vast amount of clinical data that exist in electronic medical records without causing information overload or interfering with provider thought processes is a challenge. To support the transformation of data into information and knowledge, effective electronic displays must be flexible and guide physicians' thought processes. Applying research from cognitive science and human factors engineering offers promise in improving the electronic display of clinical information. OBJECTIVES: After completing this article, readers should be able to: Appreciate the importance of supporting provider thought processes during both data entry and data review.Recognize that information does not need to be displayed and reviewed in the same way the data are entered.

Documentation of contact precautions in an electronic health record

Contact precautions are implemented to reduce transmission of multidrug-resistant organisms but may also increase hospital costs and patient complications. The goal of this study was to determine the prevalence of documentation of contact precautions (provider orders and nursing flowsheet documentation) in an electronic health record. Orders and nursing documentation were simultaneously present for only 42.3% of patient rooms with contact precaution signs, and 17.8% of rooms with signs had neither orders nor nursing documentation.

A comparison of methods to detect urinary tract infections using electronic data

BACKGROUND

The use of electronic medical records to identify common health care-associated infections (HAIs), including pneumonia, surgical site infections, bloodstream infections, and urinary tract infections (UTIs), has been proposed to help perform HAI surveillance and guide infection prevention efforts. Increased attention on HAIs has led to public health reporting requirements and a focus on quality improvement activities around HAIs. Traditional surveillance to detect HAIs and focus prevention efforts is labor intensive, and computer algorithms could be useful to screen electronic data and provide actionable information.

METHODS

Seven computer-based decision rules to identify UTIs were compared in a sample of 33,834 admissions to an urban academic health center. These decision rules included combinations of laboratory data, patient clinical data, and administrative data (for example, International Statistical Classification of Diseases and Related Health Problems, Ninth Revision [ICD-9] codes).

RESULTS

Of 33,834 hospital admissions, 3,870 UTIs were identified by at least one of the decision rules. The use of ICD-9 codes alone identified 2,614 UTIs. Laboratory-based definitions identified 2,773 infections, but when the presence of fever was included, only 1,125 UTIs were identified. The estimated sensitivity of ICD-9 codes was 55.6% (95% confidence interval [CI], 52.5%-58.5%) when compared with a culture- and symptom-based definition. Of the UTIs identified by ICD-9 codes, 167/1,125 (14.8%) also met two urine-culture decision rules.

DISCUSSION

Use of the example of UTI identification shows how different algorithms may be appropriate, depending on the goal of case identification. Electronic surveillance methods may be beneficial for mandatory reporting, process improvement, and economic analysis.

Effects of blood glucose transcription mismatches on a computer-based intensive insulin therapy protocol

PURPOSE

Computerized clinical decision support systems (CDSS) for intensive insulin therapy (IIT) generate recommendations using blood glucose (BG) values manually transcribed from testing devices to computers, a potential source of error. We quantified the frequency and effect of blood glucose transcription mismatches on IIT protocol performance.

METHODS

We examined 38 months of retrospective data for patients treated with CDSS IIT in two intensive care units at one teaching hospital. A manually transcribed BG value not equal to a corresponding device value was deemed mismatched. For mismatches we recalculated CDSS recommendations using device BG values. We compared matched and mismatched data in terms of CDSS alerts, blood glucose variability, and dosing.

RESULTS

Of 189,499 CDSS IIT instances, 5.3% contained mismatched BG values. Mismatched data triggered 93 false alerts and failed to issue 170 alerts for nurses to notify physicians. Four of six BG variability measures differed between matched and mismatched data. Overall insulin dose was greater for matched than mismatched [matched 3.8 (1.6-6.0), median (interquartile range, IQR), versus 3.6 (1.6-5.7); p < 0.001], but recalculated and actual dose were similar. In mismatches preceding hypoglycemia, recalculated insulin dose was significantly lower than actual dose [recalculated 2.7 (0.4-5.0), median (IQR), versus 3.5 (1.4-5.6)]. In mismatches preceding hyperglycemia, recalculated insulin dose was significantly greater than actual dose [recalculated 4.7 (3.3-6.2), median (IQR), versus 3.3 (2.4-4.3); p < 0.001]. Administration of recalculated doses might have prevented blood glucose excursions.

CONCLUSIONS

Mismatched blood glucose values can influence CDSS IIT protocol performance.

Social, organizational, and contextual characteristics of clinical decision support systems for intensive insulin therapy: a literature review and case study

INTRODUCTION

Evaluations of computerized clinical decision support systems (CDSS) typically focus on clinical performance changes and do not include social, organizational, and contextual characteristics explaining use and effectiveness. Studies of CDSS for intensive insulin therapy (IIT) are no exception, and the literature lacks an understanding of effective computer-based IIT implementation and operation.

RESULTS

This paper presents (1) a literature review of computer-based IIT evaluations through the lens of institutional theory, a discipline from sociology and organization studies, to demonstrate the inconsistent reporting of workflow and care process execution and (2) a single-site case study to illustrate how computer-based IIT requires substantial organizational change and creates additional complexity with unintended consequences including error.

DISCUSSION

Computer-based IIT requires organizational commitment and attention to site-specific technology, workflow, and care processes to achieve intensive insulin therapy goals. The complex interaction between clinicians, blood glucose testing devices, and CDSS may contribute to workflow inefficiency and error. Evaluations rarely focus on the perspective of nurses, the primary users of computer-based IIT whose knowledge can potentially lead to process and care improvements.

CONCLUSION

This paper addresses a gap in the literature concerning the social, organizational, and contextual characteristics of CDSS in general and for intensive insulin therapy specifically. Additionally, this paper identifies areas for future research to define optimal computer-based IIT process execution: the frequency and effect of manual data entry error of blood glucose values, the frequency and effect of nurse overrides of CDSS insulin dosing recommendations, and comprehensive ethnographic study of CDSS for IIT.

How to implement information technology in the operating room and the intensive care unit

The number of operating rooms and intensive care units looking for a data management system to perform their increasingly complex tasks is rising. Although at this time only a minority is computerized, within the next few years many centres will start implementing information technology. The transition towards a computerized system is a major venture, which will have a major impact on workflow. This chapter reviews the present literature. Published papers on this subject are predominantly single- or multi-centre implementation reports. The general principles that should guide such a process are described. For healthcare institutions or individual practitioners that plan to undertake this venture, the implementation process is described in a practical, nine-step overview.

Robust parameter extraction for decision support using multimodal intensive care data

Digital information flow within the intensive care unit (ICU) continues to grow, with advances in technology and computational biology. Recent developments in the integration and archiving of these data have resulted in new opportunities for data analysis and clinical feedback. New problems associated with ICU databases have also arisen. ICU data are high-dimensional, often sparse, asynchronous and irregularly sampled, as well as being non-stationary, noisy and subject to frequent exogenous perturbations by clinical staff. Relationships between different physiological parameters are usually nonlinear (except within restricted ranges), and the equipment used to measure the observables is often inherently error-prone and biased. The prior probabilities associated with an individual's genetics, pre-existing conditions, lifestyle and ongoing medical treatment all affect prediction and classification accuracy. In this paper, we describe some of the key problems and associated methods that hold promise for robust parameter extraction and data fusion for use in clinical decision support in the ICU.

Validation of knowledge acquisition for surgical process models

OBJECTIVE

Surgical Process Models (SPMs) are models of surgical interventions. The objectives of this study are to validate acquisition methods for Surgical Process Models and to assess the performance of different observer populations.

DESIGN

The study examined 180 SPM of simulated Functional Endoscopic Sinus Surgeries (FESS), recorded with observation software. About 150,000 single measurements in total were analyzed.

MEASUREMENTS

Validation metrics were used for assessing the granularity, content accuracy, and temporal accuracy of structures of SPMs.

RESULTS

Differences between live observations and video observations are not statistically significant. Observations performed by subjects with medical backgrounds gave better results than observations performed by subjects with technical backgrounds. Granularity was reconstructed correctly by 90%, content by 91%, and the mean temporal accuracy was 1.8 s.

CONCLUSION

The study shows the validity of video as well as live observations for modeling Surgical Process Models. For routine use, the authors recommend live observations due to their flexibility and effectiveness. If high precision is needed or the SPM parameters are altered during the study, video observations are the preferable approach.