Deployment of artificial intelligence for radiographic diagnosis of COVID-19 pneumonia in the emergency department

Robust Stochastic Neural Ensemble Learning With Noisy Labels for Thoracic Disease Classification

Chest radiography is the most common radiology examination for thoracic disease diagnosis, such as pneumonia. A tremendous number of chest X-rays prompt data-driven deep learning models in constructing computer-aided diagnosis systems for thoracic diseases. However, in realistic radiology practice, a deep learning-based model often suffers from performance degradation when trained on data with noisy labels possibly caused by different types of annotation biases. To this end, we present a novel stochastic neural ensemble learning (SNEL) framework for robust thoracic disease diagnosis using chest X-rays. The core idea of our method is to learn from noisy labels by constructing model ensembles and designing noise-robust loss functions. Specifically, we propose a fast neural ensemble method that collects parameters simultaneously across model instances and along optimization trajectories. Moreover, we propose a loss function that both optimizes a robust measure and characterizes a diversity measure of ensembles. We evaluated our proposed SNEL method on three publicly available hospital-scale chest X-ray datasets. The experimental results indicate that our method outperforms competing methods and demonstrate the effectiveness and robustness of our method in learning from noisy labels. Our code is available at https://github.com/hywang01/SNEL.

Impact of a deep learning sepsis prediction model on quality of care and survival

Sepsis remains a major cause of mortality and morbidity worldwide. Algorithms that assist with the early recognition of sepsis may improve outcomes, but relatively few studies have examined their impact on real-world patient outcomes. Our objective was to assess the impact of a deep-learning model (COMPOSER) for the early prediction of sepsis on patient outcomes. We completed a before-and-after quasi-experimental study at two distinct Emergency Departments (EDs) within the UC San Diego Health System. We included 6217 adult septic patients from 1/1/2021 through 4/30/2023. The exposure tested was a nurse-facing Best Practice Advisory (BPA) triggered by COMPOSER. In-hospital mortality, sepsis bundle compliance, 72-h change in sequential organ failure assessment (SOFA) score following sepsis onset, ICU-free days, and the number of ICU encounters were evaluated in the pre-intervention period (705 days) and the post-intervention period (145 days). The causal impact analysis was performed using a Bayesian structural time-series approach with confounder adjustments to assess the significance of the exposure at the 95% confidence level. The deployment of COMPOSER was significantly associated with a 1.9% absolute reduction (17% relative decrease) in in-hospital sepsis mortality (95% CI, 0.3%-3.5%), a 5.0% absolute increase (10% relative increase) in sepsis bundle compliance (95% CI, 2.4%-8.0%), and a 4% (95% CI, 1.1%-7.1%) reduction in 72-h SOFA change after sepsis onset in causal inference analysis. This study suggests that the deployment of COMPOSER for early prediction of sepsis was associated with a significant reduction in mortality and a significant increase in sepsis bundle compliance.

Making the Improbable Possible: Generalizing Models Designed for a Syndrome-Based, Heterogeneous Patient Landscape

Syndromic conditions, such as sepsis, are commonly encountered in the intensive care unit. Although these conditions are easy for clinicians to grasp, these conditions may limit the performance of machine-learning algorithms. Individual hospital practice patterns may limit external generalizability. Data missingness is another barrier to optimal algorithm performance and various strategies exist to mitigate this. Recent advances in data science, such as transfer learning, conformal prediction, and continual learning, may improve generalizability of machine-learning algorithms in critically ill patients. Randomized trials with these approaches are indicated to demonstrate improvements in patient-centered outcomes at this point.

Role of Imaging in the Management of Patients with SARS-CoV-2 Lung Involvement Admitted to the Emergency Department: A Systematic Review

During the waves of the coronavirus disease (COVID-19) pandemic, emergency departments were overflowing with patients suffering with suspected medical or surgical issues. In these settings, healthcare staff should be able to deal with different medical and surgical scenarios while protecting themselves against the risk of contamination. Various strategies were used to overcome the most critical issues and guarantee quick and efficient diagnostic and therapeutic charts. The use of saliva and nasopharyngeal swab Nucleic Acid Amplification Tests (NAAT) in the diagnosis of COVID-19 was one of the most adopted worldwide. However, NAAT results were slow to report and could sometimes create significant delays in patient management, especially during pandemic peaks. On these bases, radiology has played and continues to play an essential role in detecting COVID-19 patients and solving differential diagnosis between different medical conditions. This systematic review aims to summarize the role of radiology in the management of COVID-19 patients admitted to emergency departments by using chest X-rays (CXR), computed tomography (CT), lung ultrasounds (LUS), and artificial intelligence (AI).

Artificial Intelligence Functionalities During the COVID-19 Pandemic

BACKGROUND

The coronavirus disease 2019 (COVID-19) pandemic has led us to use virtual solutions and emerging technologies such as artificial intelligence (AI). Recent studies have clearly demonstrated the role of AI in health care and medical practice; however, a comprehensive review can identify potential yet not fulfilled functionalities of such technologies in pandemics. Therefore, this scoping review study aims at assessing AI functionalities in the COVID-19 pandemic in 2022.

METHODS

A systematic search was carried out in PubMed, Cochran Library, Scopus, Science Direct, ProQuest, and Web of Science from 2019 to May 9, 2022. Researchers selected the articles according to the search keywords. Finally, the articles mentioning the functionalities of AI in the COVID-19 pandemic were evaluated. Two investigators performed this process.

RESULTS

Initial search resulted in 9123 articles. After reviewing the title, abstract, and full text of these articles, and applying the inclusion and exclusion criteria, 4 articles were selectd for the final analysis. All 4 were cross-sectional studies. Two studies (50%) were performed in the United States, 1 (25%) in Israel, and 1 (25%) in Saudi Arabia. They covered the functionalities of AI in the prediction, detection, and diagnosis of COVID-19.

CONCLUSIONS

To the extent of the researchers' knowledge, this study is the first scoping review that assesses the AI functionalities in the COVID-19 pandemic. Health-care organizations need decision support technologies and evidence-based apparatuses that can perceive, think, and reason not dissimilar to human beings. Potential functionalities of such technologies can be used to predict mortality, detect, screen, and trace current and former patients, analyze health data, prioritize high-risk patients, and better allocate hospital resources in pandemics, and generally in health-care settings.

Performance of a Chest Radiograph AI Diagnostic Tool for COVID-19: A Prospective Observational Study

Purpose

To conduct a prospective observational study across 12 U.S. hospitals to evaluate real-time performance of an interpretable artificial intelligence (AI) model to detect COVID-19 on chest radiographs.

Materials and Methods

A total of 95 363 chest radiographs were included in model training, external validation, and real-time validation. The model was deployed as a clinical decision support system, and performance was prospectively evaluated. There were 5335 total real-time predictions and a COVID-19 prevalence of 4.8% (258 of 5335). Model performance was assessed with use of receiver operating characteristic analysis, precision-recall curves, and F1 score. Logistic regression was used to evaluate the association of race and sex with AI model diagnostic accuracy. To compare model accuracy with the performance of board-certified radiologists, a third dataset of 1638 images was read independently by two radiologists.

Results

Participants positive for COVID-19 had higher COVID-19 diagnostic scores than participants negative for COVID-19 (median, 0.1 [IQR, 0.0-0.8] vs 0.0 [IQR, 0.0-0.1], respectively; P < .001). Real-time model performance was unchanged over 19 weeks of implementation (area under the receiver operating characteristic curve, 0.70; 95% CI: 0.66, 0.73). Model sensitivity was higher in men than women (P = .01), whereas model specificity was higher in women (P = .001). Sensitivity was higher for Asian (P = .002) and Black (P = .046) participants compared with White participants. The COVID-19 AI diagnostic system had worse accuracy (63.5% correct) compared with radiologist predictions (radiologist 1 = 67.8% correct, radiologist 2 = 68.6% correct; McNemar P < .001 for both).

Conclusion

AI-based tools have not yet reached full diagnostic potential for COVID-19 and underperform compared with radiologist prediction.Keywords: Diagnosis, Classification, Application Domain, Infection, Lung Supplemental material is available for this article.. © RSNA, 2022.

Artificial Intelligence-Based Detection of Pneumonia in Chest Radiographs

Artificial intelligence is gaining increasing relevance in the field of radiology. This study retrospectively evaluates how a commercially available deep learning algorithm can detect pneumonia in chest radiographs (CR) in emergency departments. The chest radiographs of 948 patients with dyspnea between 3 February and 8 May 2020, as well as 15 October and 15 December 2020, were used. A deep learning algorithm was used to identify opacifications associated with pneumonia, and the performance was evaluated by using ROC analysis, sensitivity, specificity, PPV and NPV. Two radiologists assessed all enrolled images for pulmonal infection patterns as the reference standard. If consolidations or opacifications were present, the radiologists classified the pulmonal findings regarding a possible COVID-19 infection because of the ongoing pandemic. The AUROC value of the deep learning algorithm reached 0.923 when detecting pneumonia in chest radiographs with a sensitivity of 95.4%, specificity of 66.0%, PPV of 80.2% and NPV of 90.8%. The detection of COVID-19 pneumonia in CR by radiologists was achieved with a sensitivity of 50.6% and a specificity of 73%. The deep learning algorithm proved to be an excellent tool for detecting pneumonia in chest radiographs. Thus, the assessment of suspicious chest radiographs can be purposefully supported, shortening the turnaround time for reporting relevant findings and aiding early triage.

Speech as a Biomarker for COVID-19 Detection Using Machine Learning

The use of speech as a biomedical signal for diagnosing COVID-19 is investigated using statistical analysis of speech spectral features and classification algorithms based on machine learning. It is established that spectral features of speech, obtained by computing the short-time Fourier Transform (STFT), get altered in a statistical sense as a result of physiological changes. These spectral features are then used as input features to machine learning-based classification algorithms to classify them as coming from a COVID-19 positive individual or not. Speech samples from healthy as well as “asymptomatic” COVID-19 positive individuals have been used in this study. It is shown that the RMS error of statistical distribution fitting is higher in the case of speech samples of COVID-19 positive speech samples as compared to the speech samples of healthy individuals. Five state-of-the-art machine learning classification algorithms have also been analyzed, and the performance evaluation metrics of these algorithms are also presented. The tuning of machine learning model parameters is done so as to minimize the misclassification of COVID-19 positive individuals as being COVID-19 negative since the cost associated with this misclassification is higher than the opposite misclassification. The best performance in terms of the “recall” metric is observed for the Decision Forest algorithm which gives a recall value of 0.7892.

Radiologist-supervised Transfer Learning: Improving Radiographic Localization of Pneumonia and Prognostication of Patients With COVID-19

PURPOSE

To assess the potential of a transfer learning strategy leveraging radiologist supervision to enhance convolutional neural network-based (CNN) localization of pneumonia on radiographs and to further assess the prognostic value of CNN severity quantification on patients evaluated for COVID-19 pneumonia, for whom severity on the presenting radiograph is a known predictor of mortality and intubation.

MATERIALS AND METHODS

We obtained an initial CNN previously trained to localize pneumonia along with 25,684 radiographs used for its training. We additionally curated 1466 radiographs from patients who had a computed tomography (CT) performed on the same day. Regional likelihoods of pneumonia were then annotated by cardiothoracic radiologists, referencing these CTs. Combining data, a preexisting CNN was fine-tuned using transfer learning. Whole-image and regional performance of the updated CNN was assessed using receiver-operating characteristic area under the curve and Dice. Finally, the value of CNN measurements was assessed with survival analysis on 203 patients with COVID-19 and compared against modified radiographic assessment of lung edema (mRALE) score.

RESULTS

Pneumonia detection area under the curve improved on both internal (0.756 to 0.841) and external (0.864 to 0.876) validation data. Dice overlap also improved, particularly in the lung bases (R: 0.121 to 0.433, L: 0.111 to 0.486). There was strong correlation between radiologist mRALE score and CNN fractional area of involvement (ρ=0.85). Survival analysis showed similar, strong prognostic ability of the CNN and mRALE for mortality, likelihood of intubation, and duration of hospitalization among patients with COVID-19.

CONCLUSIONS

Radiologist-supervised transfer learning can enhance the ability of CNNs to localize and quantify the severity of disease. Closed-loop systems incorporating radiologists may be beneficial for continued improvement of artificial intelligence algorithms.

The application of a deep learning system developed to reduce the time for RT-PCR in COVID-19 detection

Reducing the time to diagnose COVID-19 helps to manage insufficient isolation-bed resources and adequately accommodate critically ill patients. There is currently no alternative method to real-time reverse transcriptase polymerase chain reaction (RT-PCR), which requires 40 cycles to diagnose COVID-19. We propose a deep learning (DL) model to improve the speed of COVID-19 RT-PCR diagnosis. We developed and tested a DL model using the long short-term memory method with a dataset of fluorescence values measured in each cycle of 5810 RT-PCR tests. Among the DL models developed here, the diagnostic performance of the 21st model showed an area under the receiver operating characteristic (AUROC), sensitivity, and specificity of 84.55%, 93.33%, and 75.72%, respectively. The diagnostic performance of the 24th model showed an AUROC, sensitivity, and specificity of 91.27%, 90.00%, and 92.54%, respectively.

Process Improvement Approaches for Increasing the Response of Emergency Departments against the COVID-19 Pandemic: A Systematic Review

The COVID-19 pandemic has strongly affected the dynamics of Emergency Departments (EDs) worldwide and has accentuated the need for tackling different operational inefficiencies that decrease the quality of care provided to infected patients. The EDs continue to struggle against this outbreak by implementing strategies maximizing their performance within an uncertain healthcare environment. The efforts, however, have remained insufficient in view of the growing number of admissions and increased severity of the coronavirus disease. Therefore, the primary aim of this paper is to review the literature on process improvement interventions focused on increasing the ED response to the current COVID-19 outbreak to delineate future research lines based on the gaps detected in the practical scenario. Therefore, we applied the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines to perform a review containing the research papers published between December 2019 and April 2021 using ISI Web of Science, Scopus, PubMed, IEEE, Google Scholar, and Science Direct databases. The articles were further classified taking into account the research domain, primary aim, journal, and publication year. A total of 65 papers disseminated in 51 journals were concluded to satisfy the inclusion criteria. Our review found that most applications have been directed towards predicting the health outcomes in COVID-19 patients through machine learning and data analytics techniques. In the overarching pandemic, healthcare decision makers are strongly recommended to integrate artificial intelligence techniques with approaches from the operations research (OR) and quality management domains to upgrade the ED performance under social-economic restrictions.

The Clinical Information Systems Response to the COVID-19 Pandemic

OBJECTIVE

The year 2020 was predominated by the coronavirus disease 2019 (COVID-19) pandemic. The objective of this article is to review the areas in which clinical information systems (CIS) can be and have been utilized to support and enhance the response of healthcare systems to pandemics, focusing on COVID-19.

METHODS

PubMed/MEDLINE, Google Scholar, the tables of contents of major informatics journals, and the bibliographies of articles were searched for studies pertaining to CIS, pandemics, and COVID-19 through October 2020. The most informative and detailed studies were highlighted, while many others were referenced.

RESULTS

CIS were heavily relied upon by health systems and governmental agencies worldwide in response to COVID-19. Technology-based screening tools were developed to assist rapid case identification and appropriate triaging. Clinical care was supported by utilizing the electronic health record (EHR) to onboard frontline providers to new protocols, offer clinical decision support, and improve systems for diagnostic testing. Telehealth became the most rapidly adopted medical trend in recent history and an essential strategy for allowing safe and effective access to medical care. Artificial intelligence and machine learning algorithms were developed to enhance screening, diagnostic imaging, and predictive analytics - though evidence of improved outcomes remains limited. Geographic information systems and big data enabled real-time dashboards vital for epidemic monitoring, hospital preparedness strategies, and health policy decision making. Digital contact tracing systems were implemented to assist a labor-intensive task with the aim of curbing transmission. Large scale data sharing, effective health information exchange, and interoperability of EHRs remain challenges for the informatics community with immense clinical and academic potential. CIS must be used in combination with engaged stakeholders and operational change management in order to meaningfully improve patient outcomes.

CONCLUSION

Managing a pandemic requires widespread, timely, and effective distribution of reliable information. In the past year, CIS and informaticists made prominent and influential contributions in the global response to the COVID-19 pandemic.

Review of Current COVID-19 Diagnostics and Opportunities for Further Development

Diagnostic testing plays a critical role in addressing the coronavirus disease 2019 (COVID-19) pandemic, caused by Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2). Rapid and accurate diagnostic tests are imperative for identifying and managing infected individuals, contact tracing, epidemiologic characterization, and public health decision making. Laboratory testing may be performed based on symptomatic presentation or for screening of asymptomatic people. Confirmation of SARS-CoV-2 infection is typically by nucleic acid amplification tests (NAAT), which requires specialized equipment and training and may be particularly challenging in resource-limited settings. NAAT may give false-negative results due to timing of sample collection relative to infection, improper sampling of respiratory specimens, inadequate preservation of samples, and technical limitations; false-positives may occur due to technical errors, particularly contamination during the manual real-time polymerase chain reaction (RT-PCR) process. Thus, clinical presentation, contact history and contemporary phyloepidemiology must be considered when interpreting results. Several sample-to-answer platforms, including high-throughput systems and Point of Care (PoC) assays, have been developed to increase testing capacity and decrease technical errors. Alternatives to RT-PCR assay, such as other RNA detection methods and antigen tests may be appropriate for certain situations, such as resource-limited settings. While sequencing is important to monitor on-going evolution of the SARS-CoV-2 genome, antibody assays are useful for epidemiologic purposes. The ever-expanding assortment of tests, with varying clinical utility, performance requirements, and limitations, merits comparative evaluation. We herein provide a comprehensive review of currently available COVID-19 diagnostics, exploring their pros and cons as well as appropriate indications. Strategies to further optimize safety, speed, and ease of SARS-CoV-2 testing without compromising accuracy are suggested. Access to scalable diagnostic tools and continued technologic advances, including machine learning and smartphone integration, will facilitate control of the current pandemic as well as preparedness for the next one.