Validation of the ICU-DaMa tool for automatically extracting variables for minimum dataset and quality indicators: The importance of data quality assessment

Advancing ICU patient care with a Real-Time predictive model for mechanical Power to mitigate VILI

BACKGROUND

Invasive Mechanical Ventilation (IMV) in Intensive Care Units (ICU) significantly increases the risk of Ventilator-Induced Lung Injury (VILI), necessitating careful management of mechanical power (MP). This study aims to develop a real-time predictive model of MP utilizing Artificial Intelligence to mitigate VILI.

METHODOLOGY

A retrospective observational study was conducted, extracting patient data from Clinical Information Systems from 2018 to 2022. Patients over 18 years old with more than 6 h of IMV were selected. Continuous data on IMV variables, laboratory data, monitoring, procedures, demographic data, type of admission, reason for admission, and APACHE II at admission were extracted. The variables with the highest correlation to MP were used for prediction and IMV data was grouped in 15-minute intervals using the mean. A mixed neural network model was developed to forecast MP 15 min in advance, using IMV data from 6 h before the prediction and current patient status. The model's ability to predict future MP was analyzed and compared to a baseline model predicting the future value of MP as equal to the current value.

RESULTS

The cohort consisted of 1967 patients after applying inclusion criteria, with a median age of 63 years and 66.9 % male. The deep learning model achieved a mean squared error of 2.79 in the test set, indicating a 20 % improvement over the baseline model. It demonstrated high accuracy (94 %) in predicting whether MP would exceed a critical threshold of 18 J/min, which correlates with increased mortality. The integration of this model into a web platform allows clinicians real-time access to MP predictions, facilitating timely adjustments to ventilation settings.

CONCLUSIONS

The study successfully developed and integrated in clinical practice a predictive model for MP. This model will assist clinicians allowing for the adjustment of ventilatory parameters before lung damage occurs.

Quality Indicators in Adult Critical Care Medicine

Quality indicators are increasingly used in the intensive care unit (ICU) to compare and improve the quality of delivered healthcare. Numerous indicators have been developed and are related to multiple domains, most importantly patient safety, care timeliness and effectiveness, staff well-being, and patient/family-centered outcomes and satisfaction. In this review, we describe pertinent ICU quality indicators that are related to organizational structure (such as the availability of an intensivist 24/7 and the nurse-to-patient ratio), processes of care (such as ventilator care bundle), and outcomes (such as ICU-acquired infections and standardized mortality rate). We also present an example of a quality improvement project in an ICU indicating the steps taken to attain the desired changes in quality measures.

Assessing the impact of real-time random safety audits through full propensity score matching on reliable data from the clinical information system

BACKGROUND

Evidence-based care processes are not always applied at the bedside in critically ill patients. Numerous studies have assessed the impact of checklists and related strategies on the process of care and patient outcomes. We aimed to evaluate the effects of real-time random safety audits on process-of-care and outcome variables in critical care patients.

METHODS

This prospective study used data from the clinical information system to evaluate the impact of real-time random safety audits targeting 32 safety measures in two intensive care units during a 9-month period. We compared endpoints between patients attended with safety audits and those not attended with safety audits. The primary endpoint was mortality, measured by Cox hazard regression after full propensity-score matching. Secondary endpoints were the impact on adherence to process-of-care measures and on quality indicators.

RESULTS

We included 871 patients; 228 of these were attended in ≥ 1 real-time random safety audits. Safety audits were carried out on 390 patient-days; most improvements in the process of care were observed in safety measures related to mechanical ventilation, renal function and therapies, nutrition, and clinical information system. Although the group of patients attended in safety audits had more severe disease at ICU admission [APACHE II score 21 (16-27) vs. 20 (15-25), p = 0.023]; included a higher proportion of surgical patients [37.3 % vs. 26.4 %, p = 0.003] and a higher proportion of mechanically ventilated patients [72.8 % vs. 40.3 %, p < 0.001]; averaged more days on mechanical ventilation, central venous catheter, and urinary catheter; and had a longer ICU stay [12.5 (5.5-23.3) vs. 2.9 (1.7-5.9), p < 0.001], ICU mortality did not differ significantly between groups (19.3 % vs. 18.8 % in the group without safety rounds). After full propensity-score matching, Cox hazard regression analysis showed real-time random safety audits were associated with a lower risk of mortality throughout the ICU stay (HR 0.31; 95 %CI 0.20-0.47).

CONCLUSIONS

Real-time random safety audits are associated with a reduction in the risk of ICU mortality. Exploiting data from the clinical information system is useful in assessing the impact of them on the care process, quality indicators, and mortality.

Data Quality in Health Research: Integrative Literature Review

BACKGROUND

Decision-making and strategies to improve service delivery must be supported by reliable health data to generate consistent evidence on health status. The data quality management process must ensure the reliability of collected data. Consequently, various methodologies to improve the quality of services are applied in the health field. At the same time, scientific research is constantly evolving to improve data quality through better reproducibility and empowerment of researchers and offers patient groups tools for secured data sharing and privacy compliance.

OBJECTIVE

Through an integrative literature review, the aim of this work was to identify and evaluate digital health technology interventions designed to support the conducting of health research based on data quality.

METHODS

A search was conducted in 6 electronic scientific databases in January 2022: PubMed, SCOPUS, Web of Science, Institute of Electrical and Electronics Engineers Digital Library, Cumulative Index of Nursing and Allied Health Literature, and Latin American and Caribbean Health Sciences Literature. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses checklist and flowchart were used to visualize the search strategy results in the databases.

RESULTS

After analyzing and extracting the outcomes of interest, 33 papers were included in the review. The studies covered the period of 2017-2021 and were conducted in 22 countries. Key findings revealed variability and a lack of consensus in assessing data quality domains and metrics. Data quality factors included the research environment, application time, and development steps. Strategies for improving data quality involved using business intelligence models, statistical analyses, data mining techniques, and qualitative approaches.

CONCLUSIONS

The main barriers to health data quality are technical, motivational, economical, political, legal, ethical, organizational, human resources, and methodological. The data quality process and techniques, from precollection to gathering, postcollection, and analysis, are critical for the final result of a study or the quality of processes and decision-making in a health care organization. The findings highlight the need for standardized practices and collaborative efforts to enhance data quality in health research. Finally, context guides decisions regarding data quality strategies and techniques.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1101/2022.05.31.22275804.

Electronic health record data quality assessment and tools: a systematic review

OBJECTIVE

We extended a 2013 literature review on electronic health record (EHR) data quality assessment approaches and tools to determine recent improvements or changes in EHR data quality assessment methodologies.

MATERIALS AND METHODS

We completed a systematic review of PubMed articles from 2013 to April 2023 that discussed the quality assessment of EHR data. We screened and reviewed papers for the dimensions and methods defined in the original 2013 manuscript. We categorized papers as data quality outcomes of interest, tools, or opinion pieces. We abstracted and defined additional themes and methods though an iterative review process.

RESULTS

We included 103 papers in the review, of which 73 were data quality outcomes of interest papers, 22 were tools, and 8 were opinion pieces. The most common dimension of data quality assessed was completeness, followed by correctness, concordance, plausibility, and currency. We abstracted conformance and bias as 2 additional dimensions of data quality and structural agreement as an additional methodology.

DISCUSSION

There has been an increase in EHR data quality assessment publications since the original 2013 review. Consistent dimensions of EHR data quality continue to be assessed across applications. Despite consistent patterns of assessment, there still does not exist a standard approach for assessing EHR data quality.

CONCLUSION

Guidelines are needed for EHR data quality assessment to improve the efficiency, transparency, comparability, and interoperability of data quality assessment. These guidelines must be both scalable and flexible. Automation could be helpful in generalizing this process.

Assessing and Improving Provider Knowledge for a Cardiothoracic Intensive Care Unit Electronic Dashboard Initiative

BACKGROUND

Electronic dashboards measure intensive care unit (ICU) performance by tracking quality indicators, especially pinpointing sub-standard metrics. This helps ICUs scrutinize and change current practices in an effort to improve failing metrics. However, its technological value is lost if end users are unaware of its importance. This results in decreased staff participation, leading to unsuccessful initiation of the dashboard. Therefore, the purpose of this project was to improve cardiothoracic ICU providers' understanding of electronic dashboards by providing an educational training bundle in preparation for an electronic dashboard initiation.

METHODS

A Likert survey assessing providers' knowledge, attitudes, skills, and application of electronic dashboards was conducted. Subsequently, an educational training bundle, consisting of a digital flier and laminated pamphlets, was made available to providers for four months. After bundle review, providers were assessed using the same pre-bundle Likert survey.

RESULTS

A comparison of summated scores from pre-bundle (mean = 38.75) and post-bundle surveys (mean = 46.13) yielded an increased summated score overall (mean = 7.38, p ≤ 0.001).

CONCLUSION

An educational bundle improved providers' understanding and increased their likelihood of using electronic dashboards upon its initiation. Further studies are needed to continue increasing staff participation such as providing specific education to navigate the interface for data retrieval and interpretation.

Digital Health Data Quality Issues: Systematic Review

BACKGROUND

The promise of digital health is principally dependent on the ability to electronically capture data that can be analyzed to improve decision-making. However, the ability to effectively harness data has proven elusive, largely because of the quality of the data captured. Despite the importance of data quality (DQ), an agreed-upon DQ taxonomy evades literature. When consolidated frameworks are developed, the dimensions are often fragmented, without consideration of the interrelationships among the dimensions or their resultant impact.

OBJECTIVE

The aim of this study was to develop a consolidated digital health DQ dimension and outcome (DQ-DO) framework to provide insights into 3 research questions: What are the dimensions of digital health DQ? How are the dimensions of digital health DQ related? and What are the impacts of digital health DQ?

METHODS

Following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, a developmental systematic literature review was conducted of peer-reviewed literature focusing on digital health DQ in predominately hospital settings. A total of 227 relevant articles were retrieved and inductively analyzed to identify digital health DQ dimensions and outcomes. The inductive analysis was performed through open coding, constant comparison, and card sorting with subject matter experts to identify digital health DQ dimensions and digital health DQ outcomes. Subsequently, a computer-assisted analysis was performed and verified by DQ experts to identify the interrelationships among the DQ dimensions and relationships between DQ dimensions and outcomes. The analysis resulted in the development of the DQ-DO framework.

RESULTS

The digital health DQ-DO framework consists of 6 dimensions of DQ, namely accessibility, accuracy, completeness, consistency, contextual validity, and currency; interrelationships among the dimensions of digital health DQ, with consistency being the most influential dimension impacting all other digital health DQ dimensions; 5 digital health DQ outcomes, namely clinical, clinician, research-related, business process, and organizational outcomes; and relationships between the digital health DQ dimensions and DQ outcomes, with the consistency and accessibility dimensions impacting all DQ outcomes.

CONCLUSIONS

The DQ-DO framework developed in this study demonstrates the complexity of digital health DQ and the necessity for reducing digital health DQ issues. The framework further provides health care executives with holistic insights into DQ issues and resultant outcomes, which can help them prioritize which DQ-related problems to tackle first.

Validation of an Automated System for the Extraction of a Wide Dataset for Clinical Studies Aimed at Improving the Early Diagnosis of Candidemia

There is increasing interest in assessing whether machine learning (ML) techniques could further improve the early diagnosis of candidemia among patients with a consistent clinical picture. The objective of the present study is to validate the accuracy of a system for the automated extraction from a hospital laboratory software of a large number of features from candidemia and/or bacteremia episodes as the first phase of the AUTO-CAND project. The manual validation was performed on a representative and randomly extracted subset of episodes of candidemia and/or bacteremia. The manual validation of the random extraction of 381 episodes of candidemia and/or bacteremia, with automated organization in structured features of laboratory and microbiological data resulted in ≥99% correct extractions (with confidence interval < ±1%) for all variables. The final automatically extracted dataset consisted of 1338 episodes of candidemia (8%), 14,112 episodes of bacteremia (90%), and 302 episodes of mixed candidemia/bacteremia (2%). The final dataset will serve to assess the performance of different ML models for the early diagnosis of candidemia in the second phase of the AUTO-CAND project.

Data Quality in Health Care: Main Concepts and Assessment Methodologies

INTRODUCTION

 In the health care environment, a huge volume of data is produced on a daily basis. However, the processes of collecting, storing, sharing, analyzing, and reporting health data usually face with numerous challenges that lead to producing incomplete, inaccurate, and untimely data. As a result, data quality issues have received more attention than before.

OBJECTIVE

 The purpose of this article is to provide an insight into the data quality definitions, dimensions, and assessment methodologies.

METHODS

 In this article, a scoping literature review approach was used to describe and summarize the main concepts related to data quality and data quality assessment methodologies. Search terms were selected to find the relevant articles published between January 1, 2012 and September 31, 2022. The retrieved articles were then reviewed and the results were reported narratively.

RESULTS

 In total, 23 papers were included in the study. According to the results, data quality dimensions were various and different methodologies were used to assess them. Most studies used quantitative methods to measure data quality dimensions either in paper-based or computer-based medical records. Only two studies investigated respondents' opinions about data quality.

CONCLUSION

 In health care, high-quality data not only are important for patient care, but also are vital for improving quality of health care services and better decision making. Therefore, using technical and nontechnical solutions as well as constant assessment and supervision is suggested to improve data quality.

Secondary use of data extracted from a clinical information system to assess the adherence of tidal volume and its impact on outcomes

OBJECTIVES

To extract data from clinical information systems to automatically calculate high-resolution quality indicators to assess adherence to recommendations for low tidal volume.

DESIGN

We devised two indicators: the percentage of time under mechanical ventilation with excessive tidal volume (>8mL/kg predicted body weight) and the percentage of patients who received appropriate tidal volume (≤8mL/kg PBW) at least 80% of the time under mechanical ventilation. We developed an algorithm to automatically calculate these indicators from clinical information system data and analyzed associations between them and patients' characteristics and outcomes.

SETTINGS

This study has been carried out in our 30-bed polyvalent intensive care unit between January 1, 2014 and November 30, 2019.

PATIENTS

All patients admitted to intensive care unit ventilated >72h were included.

INTERVENTION

Use data collected automatically from the clinical information systems to assess adherence to tidal volume recommendations and its outcomes.

MAIN VARIABLES OF INTEREST

Mechanical ventilation days, ICU length of stay and mortality.

RESULTS

Of all admitted patients, 340 met the inclusion criteria. Median percentage of time under mechanical ventilation with excessive tidal volume was 70% (23%-93%); only 22.3% of patients received appropriate tidal volume at least 80% of the time. Receiving appropriate tidal volume was associated with shorter duration of mechanical ventilation and intensive care unit stay. Patients receiving appropriate tidal volume were mostly male, younger, taller, and less severely ill. Adjusted intensive care unit mortality did not differ according to percentage of time with excessive tidal volume or to receiving appropriate tidal volume at least 80% of the time.

CONCLUSIONS

Automatic calculation of process-of-care indicators from clinical information systems high-resolution data can provide an accurate and continuous measure of adherence to recommendations. Adherence to tidal volume recommendations was associated with shorter duration of mechanical ventilation and intensive care unit stay.

Methods and measures to quantify ICU patient heterogeneity

Patients in intensive care units are heterogeneous and the daily prediction of their days to discharge (DTD) a complex task that practitioners and computers are not always able to solve satisfactorily. In order to make more precise DTD predictors, it is necessary to have tools for the analysis of the heterogeneity of the patients. Unfortunately, the number of publications in this field is almost non-existent. In order to alleviate this lack of tools, we propose four methods and their corresponding measures to quantify the heterogeneity of intensive patients in the process of determining the DTD. These new methods and measures have been tested with patients admitted over four years to a tertiary hospital in Spain. The results deepen the understanding of the intensive patient and can serve as a basis for the construction of better DTD predictors.

Automatic generation of minimum dataset and quality indicators from data collected routinely by the clinical information system in an intensive care unit

BACKGROUND

Quality indicators (QIs) are being increasingly used in medicine to compare and improve the quality of care delivered. The feasibility of data collection is an important prerequisite for QIs. Information technology can improve efforts to measure processes and outcomes. In intensive care units (ICU), QIs can be automatically measured by exploiting data from clinical information systems (CIS).

OBJECTIVE

To describe the development and application of a tool to automatically generate a minimum dataset (MDS) and a set of ICU quality metrics from CIS data.

METHODS

We used the definitions for MDS and QIs proposed by the Spanish Society of Critical Care Medicine and Coronary Units. Our tool uses an extraction, transform, and load process implemented with Python to extract data stored in various tables in the CIS database and create a new associative database. This new database is uploaded to Qlik Sense, which constructs the MDS and calculates the QIs by applying the required metrics. The tool was tested using data from patients attended in a 30-bed polyvalent ICU during a six-year period.

RESULTS

We describe the definitions and metrics, and we report the MDS and QI measurements obtained through the analysis of 4546 admissions. The results show that our ICU's performance on the QIs analyzed meets the standards proposed by our national scientific society.

CONCLUSIONS

This is the first step toward using a tool to automatically obtain a set of actionable QIs to monitor and improve the quality of care in ICUs, eliminating the need for professionals to enter data manually, thus saving time and ensuring data quality.

Temporal variability analysis reveals biases in electronic health records due to hospital process reengineering interventions over seven years

OBJECTIVE

To evaluate the effects of Process-Reengineering interventions on the Electronic Health Records (EHR) of a hospital over 7 years.

MATERIALS AND METHODS

Temporal Variability Assessment (TVA) based on probabilistic data quality assessment was applied to the historic monthly-batched admission data of Hospital La Fe Valencia, Spain from 2010 to 2016. Routine healthcare data with a complete EHR was expanded by processed variables such as the Charlson Comorbidity Index.

RESULTS

Four Process-Reengineering interventions were detected by quantifiable effects on the EHR: (1) the hospital relocation in 2011 involved progressive reduction of admissions during the next four months, (2) the hospital services re-configuration incremented the number of inter-services transfers, (3) the care-services re-distribution led to transfers between facilities (4) the assignment to the hospital of a new area with 80,000 patients in 2015 inspired the discharge to home for follow up and the update of the pre-surgery planned admissions protocol that produced a significant decrease of the patient length of stay.

DISCUSSION

TVA provides an indicator of the effect of process re-engineering interventions on healthcare practice. Evaluating the effect of facilities' relocation and increment of citizens (findings 1, 3-4), the impact of strategies (findings 2-3), and gradual changes in protocols (finding 4) may help on the hospital management by optimizing interventions based on their effect on EHRs or on data reuse.

CONCLUSIONS

The effects on hospitals EHR due to process re-engineering interventions can be evaluated using the TVA methodology. Being aware of conditioned variations in EHR is of the utmost importance for the reliable reuse of routine hospitalization data.

Adapting data management education to support clinical research projects in an academic medical center

Background

Librarians and researchers alike have long identified research data management (RDM) training as a need in biomedical research. Despite the wealth of libraries offering RDM education to their communities, clinical research is an area that has not been targeted. Clinical RDM (CRDM) is seen by its community as an essential part of the research process where established guidelines exist, yet educational initiatives in this area are unknown.

Case Presentation

Leveraging my academic library's experience supporting CRDM through informationist grants and REDCap training in our medical center, I developed a 1.5 hour CRDM workshop. This workshop was designed to use established CRDM guidelines in clinical research and address common questions asked by our community through the library's existing data support program. The workshop was offered to the entire medical center 4 times between November 2017 and July 2018. This case study describes the development, implementation, and evaluation of this workshop.

Conclusions

The 4 workshops were well attended and well received by the medical center community, with 99% stating that they would recommend the class to others and 98% stating that they would use what they learned in their work. Attendees also articulated how they would implement the main competencies they learned from the workshop into their work. For the library, the effort to support CRDM has led to the coordination of a larger institutional collaborative training series to educate researchers on best practices with data, as well as the formation of institution-wide policy groups to address researcher challenges with CRDM, data transfer, and data sharing.

Big data and machine learning in critical care: Opportunities for collaborative research

The introduction of clinical information systems (CIS) in Intensive Care Units (ICUs) offers the possibility of storing a huge amount of machine-ready clinical data that can be used to improve patient outcomes and the allocation of resources, as well as suggest topics for randomized clinical trials. Clinicians, however, usually lack the necessary training for the analysis of large databases. In addition, there are issues referred to patient privacy and consent, and data quality. Multidisciplinary collaboration among clinicians, data engineers, machine-learning experts, statisticians, epidemiologists and other information scientists may overcome these problems. A multidisciplinary event (Critical Care Datathon) was held in Madrid (Spain) from 1 to 3 December 2017. Under the auspices of the Spanish Critical Care Society (SEMICYUC), the event was organized by the Massachusetts Institute of Technology (MIT) Critical Data Group (Cambridge, MA, USA), the Innovation Unit and Critical Care Department of San Carlos Clinic Hospital, and the Life Supporting Technologies group of Madrid Polytechnic University. After presentations referred to big data in the critical care environment, clinicians, data scientists and other health data science enthusiasts and lawyers worked in collaboration using an anonymized database (MIMIC III). Eight groups were formed to answer different clinical research questions elaborated prior to the meeting. The event produced analyses for the questions posed and outlined several future clinical research opportunities. Foundations were laid to enable future use of ICU databases in Spain, and a timeline was established for future meetings, as an example of how big data analysis tools have tremendous potential in our field.