Automating Quality Metrics in the Era of Electronic Medical Records: Digital Signatures for Ventilator Bundle Compliance

Building Learning Healthcare Systems for Critical Care Medicine

Learning healthcare systems are an evolving way of integrating informatics, analytics, and continuous improvement into daily practice in healthcare. This article discusses strategies to build learning healthcare systems for critical care medicine.

Validation of automated data abstraction for SCCM discovery VIRUS COVID-19 registry: practical EHR export pathways (VIRUS-PEEP)

Background

The gold standard for gathering data from electronic health records (EHR) has been manual data extraction; however, this requires vast resources and personnel. Automation of this process reduces resource burdens and expands research opportunities.

Objective

This study aimed to determine the feasibility and reliability of automated data extraction in a large registry of adult COVID-19 patients.

Materials and methods

This observational study included data from sites participating in the SCCM Discovery VIRUS COVID-19 registry. Important demographic, comorbidity, and outcome variables were chosen for manual and automated extraction for the feasibility dataset. We quantified the degree of agreement with Cohen's kappa statistics for categorical variables. The sensitivity and specificity were also assessed. Correlations for continuous variables were assessed with Pearson's correlation coefficient and Bland-Altman plots. The strength of agreement was defined as almost perfect (0.81-1.00), substantial (0.61-0.80), and moderate (0.41-0.60) based on kappa statistics. Pearson correlations were classified as trivial (0.00-0.30), low (0.30-0.50), moderate (0.50-0.70), high (0.70-0.90), and extremely high (0.90-1.00).

Measurements and main results

The cohort included 652 patients from 11 sites. The agreement between manual and automated extraction for categorical variables was almost perfect in 13 (72.2%) variables (Race, Ethnicity, Sex, Coronary Artery Disease, Hypertension, Congestive Heart Failure, Asthma, Diabetes Mellitus, ICU admission rate, IMV rate, HFNC rate, ICU and Hospital Discharge Status), and substantial in five (27.8%) (COPD, CKD, Dyslipidemia/Hyperlipidemia, NIMV, and ECMO rate). The correlations were extremely high in three (42.9%) variables (age, weight, and hospital LOS) and high in four (57.1%) of the continuous variables (Height, Days to ICU admission, ICU LOS, and IMV days). The average sensitivity and specificity for the categorical data were 90.7 and 96.9%.

Conclusion and relevance

Our study confirms the feasibility and validity of an automated process to gather data from the EHR.

Automating Quality Measures for Heart Failure Using Natural Language Processing: A Descriptive Study in the Department of Veterans Affairs

Background

We developed an accurate, stakeholder-informed, automated, natural language processing (NLP) system to measure the quality of heart failure (HF) inpatient care, and explored the potential for adoption of this system within an integrated health care system.

Objective

To accurately automate a United States Department of Veterans Affairs (VA) quality measure for inpatients with HF.

Methods

We automated the HF quality measure Congestive Heart Failure Inpatient Measure 19 (CHI19) that identifies whether a given patient has left ventricular ejection fraction (LVEF) <40%, and if so, whether an angiotensin-converting enzyme inhibitor or angiotensin-receptor blocker was prescribed at discharge if there were no contraindications. We used documents from 1083 unique inpatients from eight VA medical centers to develop a reference standard (RS) to train (n=314) and test (n=769) the Congestive Heart Failure Information Extraction Framework (CHIEF). We also conducted semi-structured interviews (n=15) for stakeholder feedback on implementation of the CHIEF.

Results

The CHIEF classified each hospitalization in the test set with a sensitivity (SN) of 98.9% and positive predictive value of 98.7%, compared with an RS and SN of 98.5% for available External Peer Review Program assessments. Of the 1083 patients available for the NLP system, the CHIEF evaluated and classified 100% of cases. Stakeholders identified potential implementation facilitators and clinical uses of the CHIEF.

Conclusions

The CHIEF provided complete data for all patients in the cohort and could potentially improve the efficiency, timeliness, and utility of HF quality measurements.

Can a Novel ICU Data Display Positively Affect Patient Outcomes and Save Lives?

The aim of this study was to quantify the impact of ProCCESs AWARE, Ambient Clinical Analytics, Rochester, MN, a novel acute care electronic medical record interface, on a range of care process and patient health outcome metrics in intensive care units (ICUs). ProCCESs AWARE is a novel acute care EMR interface that contains built-in tools for error prevention, practice surveillance, decision support and reporting. We compared outcomes before and after AWARE implementation using a prospective cohort and a historical control. The study population included all critically ill adult patients (over 18 years old) admitted to four ICUs at Mayo Clinic, Rochester, MN, who stayed in hospital at least 24 h. The pre-AWARE cohort included 983 patients from 2010, and the post-AWARE cohort included 856 patients from 2014. We analyzed patient health outcomes, care process quality, and hospital charges. After adjusting for patient acuity and baseline demographics, overall in-hospital and ICU mortality odds ratios associated with AWARE intervention were 0.45 (95% confidence interval 0.30 to 0.70) and 0.38 (0.22, 0.66). ICU length of stay decreased by about 50%, hospital length of stay by 37%, and total charges for hospital stay by 30% in post AWARE cohort (by $43,745 after adjusting for patient acuity and demographics). Better organization of information in the ICU with systems like AWARE has the potential to improve important patient outcomes, such as mortality and length of stay, resulting in reductions in costs of care.

"Bundle" Practices and Ventilator-Associated Events: Not Enough

OBJECTIVE Ventilator-associated events (VAEs) are nosocomial events correlated with length of stay, costs, and mortality. Current ventilator bundle practices target the older definition of ventilator-associated pneumonia and have not been systematically evaluated for their impact on VAEs. DESIGN Retrospective cohort study. SETTING Tertiary medical center between January 2012 and August 2014. PARTICIPANTS All adult patients ventilated for at least 24 hours at our institution. INTERVENTIONS We conducted univariate analyses for compliance with each element; we focused on VAEs occurring within a 2-day window of failure to meet any ventilator bundle element. We used Cox proportional hazard models to assess the effect of stress ulcer prophylaxis, deep vein thrombosis (DVT) prophylaxis, oral care, and sedation breaks on VAEs. We adjusted models for gender, age, and Acute Physiology and Chronic Health Evaluation (APACHE) III scores. RESULTS Our cohort comprised 2,660 patients with 16,858 ventilator days and 77 VAEs. Adjusting for APACHE score and gender, only oral care was associated with a reduction in the risk of VAE (hazard ratio [HR], 0.44; 95% confidence interval [CI], 0.26-0.77). The DVT prophylaxis and sedation breaks did not show any significant impact on VAEs. Stress ulcer prophylaxis trended toward an increased risk of VAE (HR, 1.59; 95% CI, 1.00-2.56). CONCLUSION Although limited by a low baseline rate of VAEs, existing ventilator bundle practices do not appear to target VAEs well. Oral care is clearly important, but the impact of DVT prophylaxis, sedation breaks, and especially stress ulcer prophylaxis are questionable at best. Infect Control Hosp Epidemiol 2016;1453-1457.