External Validity of Electronic Sniffers for Automated Recognition of Acute Respiratory Distress Syndrome

Learning using privileged information with logistic regression on acute respiratory distress syndrome detection

The advanced learning paradigm, learning using privileged information (LUPI), leverages information in training that is not present at the time of prediction. In this study, we developed privileged logistic regression (PLR) models under the LUPI paradigm to detect acute respiratory distress syndrome (ARDS), with mechanical ventilation variables or chest x-ray image features employed in the privileged domain and electronic health records in the base domain. In model training, the objective of privileged logistic regression was designed to incorporate data from the privileged domain and encourage knowledge transfer across the privileged and base domains. An asymptotic analysis was also performed, yielding sufficient conditions under which the addition of privileged information increases the rate of convergence in the proposed model. Results for ARDS detection show that PLR models achieve better classification performances than logistic regression models trained solely on the base domain, even when privileged information is partially available. Furthermore, PLR models demonstrate performance on par with or superior to state-of-the-art models under the LUPI paradigm. As the proposed models are effective, easy to interpret, and highly explainable, they are ideal for other clinical applications where privileged information is at least partially available.

Electronic health record biobank cohort recapitulates an association between the MUC5B promoter polymorphism and ARDS in critically ill adults

Background

Large population-based DNA biobanks linked to electronic health records (EHRs) may provide advantages over traditional study designs for identifying genetic drivers of ARDS.

Research Question

Can ARDS be identified in an EHR biobank, and can this approach validate a previously reported genetic risk factor for ARDS?

Study Design and Methods

We analyzed two genotyped cohorts from one academic medical center: a prospective biomarker study of critically ill adults (VALID cohort), and hospitalized participants in a de-identified EHR biobank (BioVU). ARDS status was assessed by clinician-investigator review in VALID and an EHR-derived algorithm in BioVU (EHR-ARDS). We tested the association between the MUC5B promoter polymorphism (rs35705950) with development of ARDS/EHR-ARDS in each cohort.

Results

In VALID, 2,795 patients were included, age was 55 [43, 66] (median [IQR]) years, and 718 (25.7%) developed ARDS. In BioVU, 9,025 hospitalized participants were included, age was 60 [48, 70] , and 1,056 (11.7%) developed EHR-ARDS. We observed a significant interaction between age and rs35705950 on ARDS risk in VALID: in older patients rs35705950 was associated with increased ARDS risk (OR: 1.44; 95%CI 1.08-1.92; p=0.012) whereas among younger patients this effect was attenuated (OR: 0.84; 95%CI: 0.62-1.14; p=0.26). In BioVU, rs35705950 was associated with increased risk for EHR-ARDS among all participants (OR: 1.20; 95%CI: 1.00-1.43, p=0.043) and this relationship did not vary by age. The polymorphism was also associated with more severe oxygenation impairment among BioVU participants who required mechanical ventilation.

Interpretation

The MUC5B promoter polymorphism was associated with ARDS in two cohorts of at-risk hospitalized adults. Although age-related effect modification was observed only in the prospective biomarker cohort, the EHR cohort identified a consistent association between MUC5B and ARDS risk regardless of age and a novel association with oxygenation impairment. Our study highlights the potential for EHR biobanks to enable precision-medicine ARDS studies.

Machine Learning Tools for Acute Respiratory Distress Syndrome Detection and Prediction

Machine learning (ML) tools for acute respiratory distress syndrome (ARDS) detection and prediction are increasingly used. Therefore, understanding risks and benefits of such algorithms is relevant at the bedside. ARDS is a complex and severe lung condition that can be challenging to define precisely due to its multifactorial nature. It often arises as a response to various underlying medical conditions, such as pneumonia, sepsis, or trauma, leading to widespread inflammation in the lungs. ML has shown promising potential in supporting the recognition of ARDS in ICU patients. By analyzing a variety of clinical data, including vital signs, laboratory results, and imaging findings, ML models can identify patterns and risk factors associated with the development of ARDS. This detection and prediction could be crucial for timely interventions, diagnosis and treatment. In summary, leveraging ML for the early prediction and detection of ARDS in ICU patients holds great potential to enhance patient care, improve outcomes, and contribute to the evolving landscape of precision medicine in critical care settings. This article is a concise definitive review on artificial intelligence and ML tools for the prediction and detection of ARDS in critically ill patients.

Use of pragmatic and explanatory trial designs in acute care research: lessons from COVID-19

Unique challenges arise when conducting trials to evaluate therapies already in common clinical use, including difficulty enrolling patients owing to widespread open-label use of trial therapies and the need for large sample sizes to detect small but clinically meaningful treatment effects. Despite numerous successes in trials evaluating novel interventions such as vaccines, traditional explanatory trials have struggled to provide definitive answers to time-sensitive questions for acutely ill patients with COVID-19. Pragmatic trials, which can increase efficiency by allowing some or all trial procedures to be embedded into clinical care, are increasingly proposed as a means to evaluate therapies that are in common clinical use. In this Personal View, we use two concurrently conducted COVID-19 trials of hydroxychloroquine (the US ORCHID trial and the UK RECOVERY trial) to contrast the effects of explanatory and pragmatic trial designs on trial conduct, trial results, and the care of patients managed outside of clinical trials. In view of the potential advantages and disadvantages of explanatory and pragmatic trial designs, we make recommendations for their optimal use in the evaluation of therapies in the acute care setting.

Impact of Clinician Recognition of Acute Respiratory Distress Syndrome on Evidenced-Based Interventions in the Medical ICU

Acute respiratory distress syndrome is underrecognized in the ICU, but it remains uncertain if acute respiratory distress syndrome recognition affects evidence-based acute respiratory distress syndrome care in the modern era. We sought to determine the rate of clinician-recognized acute respiratory distress syndrome in an academic medical ICU and understand how clinician-recognized-acute respiratory distress syndrome affects clinical care and patient-centered outcomes.

DESIGN

Observational cohort study.

SETTING

Single medical ICU at an academic tertiary-care hospital.

PATIENTS

Nine hundred seventy-seven critically ill adults (381 with expert-adjudicated acute respiratory distress syndrome) enrolled from 2006 to 2015.

INTERVENTIONS

Clinician-recognized-acute respiratory distress syndrome was identified using an electronic keyword search of clinical notes in the electronic health record. We assessed the classification performance of clinician-recognized acute respiratory distress syndrome for identifying expert-adjudicated acute respiratory distress syndrome. We also compared differences in ventilator settings, diuretic prescriptions, and cumulative fluid balance between clinician-recognized acute respiratory distress syndrome and unrecognized acute respiratory distress syndrome.

MEASUREMENTS AND MAIN RESULTS

Overall, clinician-recognized-acute respiratory distress syndrome had a sensitivity of 47.5%, specificity 91.1%, positive predictive value 77.4%, and negative predictive value 73.1% for expert-adjudicated acute respiratory distress syndrome. Among the 381 expert-adjudicated acute respiratory distress syndrome cases, we did not observe any differences in ventilator tidal volumes between clinician-recognized-acute respiratory distress syndrome and unrecognized acute respiratory distress syndrome, but clinician-recognized-acute respiratory distress syndrome patients had a more negative cumulative fluid balance (mean difference, -781 mL; 95% CI, [-1,846 to +283]) and were more likely to receive diuretics (49.3% vs 35.7%, p = 0.02). There were no differences in mortality, ICU length of stay, or ventilator-free days.

CONCLUSIONS

Acute respiratory distress syndrome recognition was low in this single-center study. Although acute respiratory distress syndrome recognition was not associated with lower ventilator volumes, it was associated with differences in behaviors related to fluid management. These findings have implications for the design of future studies promoting evidence-based acute respiratory distress syndrome interventions in the ICU.

Mortality prediction for patients with acute respiratory distress syndrome based on machine learning: a population-based study

Background

Traditional scoring systems for patients' outcome prediction in intensive care units such as Oxygenation Saturation Index (OSI) and Oxygenation Index (OI) may not reliably predict the clinical prognosis of patients with acute respiratory distress syndrome (ARDS). Thus, none of them have been widely accepted for mortality prediction in ARDS. This study aimed to develop and validate a mortality prediction method for patients with ARDS based on machine learning using the Medical Information Mart for Intensive Care (MIMIC-III) and Telehealth Intensive Care Unit (eICU) Collaborative Research Database (eICU-CRD) databases.

Methods

Patients with ARDS were selected based on the Berlin definition in MIMIC-III and eICU-CRD databases. The APPS score (using age, PaO₂/FiO₂, and plateau pressure), Simplified Acute Physiology Score II (SAPS-II), Sepsis-related Organ Failure Assessment (SOFA), OSI, and OI were calculated. With MIMIC-III data, a mortality prediction model was built based on the random forest (RF) algorithm, and the performance was compared to those of existing scoring systems based on logistic regression. The performance of the proposed RF method was also validated with the combined MIMIC-III and eICU-CRD data. The performance of mortality prediction was evaluated by using the area under the receiver operating characteristics curve (AUROC) and performing calibration using the Hosmer-Lemeshow test.

Results

With the MIMIC-III dataset (308 patients, for comparisons with the existing scoring systems), the RF model predicted the in-hospital mortality, 30-day mortality, and 1-year mortality with an AUROC of 0.891, 0.883, and 0.892, respectively, which were significantly higher than those of the SAPS-II, APPS, OSI, and OI (all P<0.001). In the multi-source validation (the combined dataset of 2,235 patients in MIMIC-III and 331 patients in eICU-CRD), the RF model achieved an AUROC of 0.905 and 0.736 for predicting in-hospital mortality for the MIMIC-III and eICU-CRD datasets, respectively. The calibration plots suggested good fits for our RF model and these scoring systems for predicting mortality. The platelet count and lactate level were the strongest predictive variables for predicting in-hospital mortality.

Conclusions

Compared to the existing scoring systems, machine learning significantly improved performance for predicting ARDS mortality. Validation with multi-source datasets showed a relatively robust generalisation ability of our prediction model.

Algorithmic surveillance of ICU patients with acute respiratory distress syndrome (ASIC): protocol for a multicentre stepped-wedge cluster randomised quality improvement strategy

INTRODUCTION

The acute respiratory distress syndrome (ARDS) is a highly relevant entity in critical care with mortality rates of 40%. Despite extensive scientific efforts, outcome-relevant therapeutic measures are still insufficiently practised at the bedside. Thus, there is a clear need to adhere to early diagnosis and sufficient therapy in ARDS, assuring lower mortality and multiple organ failure.

METHODS AND ANALYSIS

In this quality improvement strategy (QIS), a decision support system as a mobile application (ASIC app), which uses available clinical real-time data, is implemented to support physicians in timely diagnosis and improvement of adherence to established guidelines in the treatment of ARDS. ASIC is conducted on 31 intensive care units (ICUs) at 8 German university hospitals. It is designed as a multicentre stepped-wedge cluster randomised QIS. ICUs are combined into 12 clusters which are randomised in 12 steps. After preparation (18 months) and a control phase of 8 months for all clusters, the first cluster enters a roll-in phase (3 months) that is followed by the actual QIS phase. The remaining clusters follow in month wise steps. The coprimary key performance indicators (KPIs) consist of the ARDS diagnostic rate and guideline adherence regarding lung-protective ventilation. Secondary KPIs include the prevalence of organ dysfunction within 28 days after diagnosis or ICU discharge, the treatment duration on ICU and the hospital mortality. Furthermore, the user acceptance and usability of new technologies in medicine are examined. To show improvements in healthcare of patients with ARDS, differences in primary and secondary KPIs between control phase and QIS will be tested.

ETHICS AND DISSEMINATION

Ethical approval was obtained from the independent Ethics Committee (EC) at the RWTH Aachen Faculty of Medicine (local EC reference number: EK 102/19) and the respective data protection officer in March 2019. The results of the ASIC QIS will be presented at conferences and published in peer-reviewed journals.

TRIAL REGISTRATION NUMBER

DRKS00014330.

Use of Machine Learning to Screen for Acute Respiratory Distress Syndrome Using Raw Ventilator Waveform Data

To develop and characterize a machine learning algorithm to discriminate acute respiratory distress syndrome from other causes of respiratory failure using only ventilator waveform data.

Design

Retrospective, observational cohort study.

Setting

Academic medical center ICU.

Patients

Adults admitted to the ICU requiring invasive mechanical ventilation, including 50 patients with acute respiratory distress syndrome and 50 patients with primary indications for mechanical ventilation other than hypoxemic respiratory failure.

Interventions

None.

Measurements and Main Results

Pressure and flow time series data from mechanical ventilation during the first 24-hours after meeting acute respiratory distress syndrome criteria (or first 24-hr of mechanical ventilation for non-acute respiratory distress syndrome patients) were processed to extract nine physiologic features. A random forest machine learning algorithm was trained to discriminate between the patients with and without acute respiratory distress syndrome. Model performance was assessed using the area under the receiver operating characteristic curve, sensitivity, specificity, positive predictive value, and negative predictive value. Analyses examined performance when the model was trained using data from the first 24 hours and tested using withheld data from either the first 24 hours (24/24 model) or 6 hours (24/6 model). Area under the receiver operating characteristic curve, sensitivity, specificity, positive predictive value, and negative predictive value were 0.88, 0.90, 0.71, 0.77, and 0.90 (24/24); and 0.89, 0.90, 0.75, 0.83, and 0.83 (24/6).

Conclusions

Use of machine learning and physiologic information derived from raw ventilator waveform data may enable acute respiratory distress syndrome screening at early time points after intubation. This approach, combined with traditional diagnostic criteria, could improve timely acute respiratory distress syndrome recognition and enable automated clinical decision support, especially in settings with limited availability of conventional diagnostic tests and electronic health records.

Supervised machine learning for the early prediction of acute respiratory distress syndrome (ARDS)

PURPOSE

Acute respiratory distress syndrome (ARDS) is a serious respiratory condition with high mortality and associated morbidity. The objective of this study is to develop and evaluate a novel application of gradient boosted tree models trained on patient health record data for the early prediction of ARDS.

MATERIALS AND METHODS

9919 patient encounters were retrospectively analyzed from the Medical Information Mart for Intensive Care III (MIMIC-III) data base. XGBoost gradient boosted tree models for early ARDS prediction were created using routinely collected clinical variables and numerical representations of radiology reports as inputs. XGBoost models were iteratively trained and validated using 10-fold cross validation.

RESULTS

On a hold-out test set, algorithm classifiers attained area under the receiver operating characteristic curve (AUROC) values of 0.905 when tested for the detection of ARDS at onset and 0.827, 0.810, and 0.790 for the prediction of ARDS at 12-, 24-, and 48-h windows prior to onset, respectively.

CONCLUSION

Supervised machine learning predictions may help predict patients with ARDS up to 48 h prior to onset.

Leveraging IoTs and Machine Learning for Patient Diagnosis and Ventilation Management in the Intensive Care Unit

Future healthcare systems will rely heavily on clinical decision support systems (CDSS) to improve the decision-making processes of clinicians. To explore the design of future CDSS, we developed a research-focused CDSS for the management of patients in the intensive care unit that leverages Internet of Things (IoT) devices capable of collecting streaming physiologic data from ventilators and other medical devices. We then created machine learning (ML) models that could analyze the collected physiologic data to determine if the ventilator was delivering potentially harmful therapy and if a deadly respiratory condition, acute respiratory distress syndrome (ARDS), was present. We also present work to aggregate these models into a mobile application that can provide responsive, real-time alerts of changes in ventilation to providers. As illustrated in the recent COVID-19 pandemic, being able to accurately predict ARDS in newly infected patients can assist in prioritizing care. We show that CDSS may be used to analyze physiologic data for clinical event recognition and automated diagnosis, and we also highlight future research avenues for hospital CDSS.

External Validation of an Acute Respiratory Distress Syndrome Prediction Model Using Radiology Reports

OBJECTIVES

Acute respiratory distress syndrome is frequently under recognized and associated with increased mortality. Previously, we developed a model that used machine learning and natural language processing of text from radiology reports to identify acute respiratory distress syndrome. The model showed improved performance in diagnosing acute respiratory distress syndrome when compared to a rule-based method. In this study, our objective was to externally validate the natural language processing model in patients from an independent hospital setting.

DESIGN

Secondary analysis of data across five prospective clinical studies.

SETTING

An urban, tertiary care, academic hospital.

PATIENTS

Adult patients admitted to the medical ICU and at-risk for acute respiratory distress syndrome.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

The natural language processing model was previously derived and internally validated in burn, trauma, and medical patients at Loyola University Medical Center. Two machine learning models were examined with the following text features from qualifying radiology reports: 1) word representations (n-grams) and 2) standardized clinical named entity mentions mapped from the National Library of Medicine Unified Medical Language System. The models were externally validated in a cohort of 235 patients at the University of Chicago Medicine, among which 110 (47%) were diagnosed with acute respiratory distress syndrome by expert annotation. During external validation, the n-gram model demonstrated good discrimination between acute respiratory distress syndrome and nonacute respiratory distress syndrome patients (C-statistic, 0.78; 95% CI, 0.72-0.84). The n-gram model had a higher discrimination for acute respiratory distress syndrome when compared with the standardized named entity model, although not statistically significant (C-statistic 0.78 vs 0.72; p = 0.09). The most important features in the model had good face validity for acute respiratory distress syndrome characteristics but differences in frequencies did occur between hospital settings.

CONCLUSIONS

Our computable phenotype for acute respiratory distress syndrome had good discrimination in external validation and may be used by other health systems for case-identification. Discrepancies in feature representation are likely due to differences in characteristics of the patient cohorts.

Electronic "Sniffer" Systems to Identify the Acute Respiratory Distress Syndrome

BACKGROUND

The acute respiratory distress syndrome (ARDS) results in substantial mortality but remains underdiagnosed in clinical practice. Automated ARDS "sniffer" systems, tools that can automatically analyze electronic medical record data, have been developed to improve recognition of ARDS in clinical practice.

OBJECTIVES

To perform a systematic review examining the evidence underlying automated sniffer systems for ARDS detection.

DATA SOURCES

MEDLINE and Scopus databases through November 2018 to identify studies of tools using routinely available clinical data to detect patients with ARDS.

DATA EXTRACTION

Study design, tool description, and diagnostic performance were extracted by two reviewers. The Quality Assessment of Diagnostic Accuracy Studies-2 was used to evaluate each study for risk of bias in four domains: patient selection, index test, reference standard, and study flow and timing.

SYNTHESIS

Among 480 studies identified, 9 met inclusion criteria, and they evaluated six unique ARDS sniffer tools. Eight studies had derivation and/or temporal validation designs, with one also evaluating the effects of implementing a tool in clinical practice. A single study performed an external validation of previously published ARDS sniffer tools. Studies reported a wide range of sensitivities (43-98%) and positive predictive values (26-90%) for detection of ARDS. Most studies had potential for high risk of bias identified in their study design, including patient selection (five of nine), reference standard (four of nine), and flow and timing (three of nine). In the single external validation without any perceived risks of biases, the performance of ARDS sniffer tools was worse.

CONCLUSIONS

Sniffer systems developed to detect ARDS had moderate to high predictive value in their derivation cohorts, although most studies had the potential for high risks of bias in study design. Methodological issues may explain some of the variability in tool performance. There remains an ongoing need for robust evaluation of ARDS sniffer systems and their impact on clinical practice. Systematic review registered with PROSPERO (CRD42015026584).