COVID-19 on Chest Radiographs: A Multireader Evaluation of an Artificial Intelligence System

Application of artificial intelligence in diagnosing COVID-19 disease symptoms on chest X-rays: A systematic review

This systematic review focuses on using artificial intelligence (AI) to detect COVID-19 infection with the help of X-ray images. Methodology: In January 2022, the authors searched PubMed, Embase and Scopus using specific medical subject headings terms and filters. All articles were independently reviewed by two reviewers. All conflicts resulting from a misunderstanding were resolved by a third independent researcher. After assessing abstracts and article usefulness, eliminating repetitions and applying inclusion and exclusion criteria, six studies were found to be qualified for this study. Results: The findings from individual studies differed due to the various approaches of the authors. Sensitivity was 72.59%-100%, specificity was 79%-99.9%, precision was 74.74%-98.7%, accuracy was 76.18%-99.81%, and the area under the curve was 95.24%-97.7%. Conclusion: AI computational models used to assess chest X-rays in the process of diagnosing COVID-19 should achieve sufficiently high sensitivity and specificity. Their results and performance should be repeatable to make them dependable for clinicians. Moreover, these additional diagnostic tools should be more affordable and faster than the currently available procedures. The performance and calculations of AI-based systems should take clinical data into account.

Sequential Organ Failure Assessment Outperforms Quantitative Chest CT Imaging Parameters for Mortality Prediction in COVID-19 ARDS

(1) Background: Respiratory insufficiency with acute respiratory distress syndrome (ARDS) and multi-organ dysfunction leads to high mortality in COVID-19 patients. In times of limited intensive care unit (ICU) resources, chest CTs became an important tool for the assessment of lung involvement and for patient triage despite uncertainties about the predictive diagnostic value. This study evaluated chest CT-based imaging parameters for their potential to predict in-hospital mortality compared to clinical scores. (2) Methods: 89 COVID-19 ICU ARDS patients requiring mechanical ventilation or continuous positive airway pressure mask ventilation were included in this single center retrospective study. AI-based lung injury assessment and measurements indicating pulmonary hypertension (PA-to-AA ratio) on admission CT, oxygenation indices, lung compliance and sequential organ failure assessment (SOFA) scores on ICU admission were assessed for their diagnostic performance to predict in-hospital mortality. (3) Results: CT severity scores and PA-to-AA ratios were not significantly associated with in-hospital mortality, whereas the SOFA score showed a significant association (p < 0.001). In ROC analysis, the SOFA score resulted in an area under the curve (AUC) for in-hospital mortality of 0.74 (95%-CI 0.63–0.85), whereas CT severity scores (0.53, 95%-CI 0.40–0.67) and PA-to-AA ratios (0.46, 95%-CI 0.34–0.58) did not yield sufficient AUCs. These results were consistent for the subgroup of more critically ill patients with moderate and severe ARDS on admission (oxygenation index <200, n = 53) with an AUC for SOFA score of 0.77 (95%-CI 0.64–0.89), compared to 0.55 (95%-CI 0.39–0.72) for CT severity scores and 0.51 (95%-CI 0.35–0.67) for PA-to-AA ratios. (4) Conclusions: Severe COVID-19 disease is not limited to lung (vessel) injury but leads to a multi-organ involvement. The findings of this study suggest that risk stratification should not solely be based on chest CT parameters but needs to include multi-organ failure assessment for COVID-19 ICU ARDS patients for optimized future patient management and resource allocation.

Artificial intelligence with deep learning in nuclear medicine and radiology

The use of deep learning in medical imaging has increased rapidly over the past few years, finding applications throughout the entire radiology pipeline, from improved scanner performance to automatic disease detection and diagnosis. These advancements have resulted in a wide variety of deep learning approaches being developed, solving unique challenges for various imaging modalities. This paper provides a review on these developments from a technical point of view, categorizing the different methodologies and summarizing their implementation. We provide an introduction to the design of neural networks and their training procedure, after which we take an extended look at their uses in medical imaging. We cover the different sections of the radiology pipeline, highlighting some influential works and discussing the merits and limitations of deep learning approaches compared to other traditional methods. As such, this review is intended to provide a broad yet concise overview for the interested reader, facilitating adoption and interdisciplinary research of deep learning in the field of medical imaging.

Toward Generalizability in the Deployment of Artificial Intelligence in Radiology: Role of Computation Stress Testing to Overcome Underspecification

The clinical deployment of artificial intelligence (AI) applications in medical imaging is perhaps the greatest challenge facing radiology in the next decade. One of the main obstacles to the incorporation of automated AI-based decision-making tools in medicine is the failure of models to generalize when deployed across institutions with heterogeneous populations and imaging protocols. The most well-understood pitfall in developing these AI models is overfitting, which has, in part, been overcome by optimizing training protocols. However, overfitting is not the only obstacle to the success and generalizability of AI. Underspecification is also a serious impediment that requires conceptual understanding and correction. It is well known that a single AI pipeline, with prescribed training and testing sets, can produce several models with various levels of generalizability. Underspecification defines the inability of the pipeline to identify whether these models have embedded the structure of the underlying system by using a test set independent of, but distributed identically, to the training set. An underspecified pipeline is unable to assess the degree to which the models will be generalizable. Stress testing is a known tool in AI that can limit underspecification and, importantly, assure broad generalizability of AI models. However, the application of stress tests is new in radiologic applications. This report describes the concept of underspecification from a radiologist perspective, discusses stress testing as a specific strategy to overcome underspecification, and explains how stress tests could be designed in radiology-by modifying medical images or stratifying testing datasets. In the upcoming years, stress tests should become in radiology the standard that crash tests have become in the automotive industry. Keywords: Computer Applications-General, Informatics, Computer-aided Diagnosis © RSNA, 2021.

Artificial intelligence for imaging-based COVID-19 detection: Systematic review comparing added value of AI versus human readers

PURPOSE

A growing number of studies have examined whether Artificial Intelligence (AI) systems can support imaging-based diagnosis of COVID-19-caused pneumonia, including both gains in diagnostic performance and speed. However, what is currently missing is a combined appreciation of studies comparing human readers and AI.

METHODS

We followed PRISMA-DTA guidelines for our systematic review, searching EMBASE, PUBMED and Scopus databases. To gain insights into the potential value of AI methods, we focused on studies comparing the performance of human readers versus AI models or versus AI-supported human readings.

RESULTS

Our search identified 1270 studies, of which 12 fulfilled specific selection criteria. Concerning diagnostic performance, in testing datasets reported sensitivity was 42-100% (human readers, n = 9 studies), 60-95% (AI systems, n = 10) and 81-98% (AI-supported readers, n = 3), whilst reported specificity was 26-100% (human readers, n = 8), 61-96% (AI systems, n = 10) and 78-99% (AI-supported readings, n = 2). One study highlighted the potential of AI-supported readings for the assessment of lung lesion burden changes, whilst two studies indicated potential time savings for detection with AI.

CONCLUSIONS

Our review indicates that AI systems or AI-supported human readings show less performance variability (interquartile range) in general, and may support the differentiation of COVID-19 pneumonia from other forms of pneumonia when used in high-prevalence and symptomatic populations. However, inconsistencies related to study design, reporting of data, areas of risk of bias, as well as limitations of statistical analyses complicate clear conclusions. We therefore support efforts for developing critical elements of study design when assessing the value of AI for diagnostic imaging.

Multireader evaluation of radiologist performance for COVID-19 detection on emergency department chest radiographs

BACKGROUND

Chest radiographs (CXR) are frequently used as a screening tool for patients with suspected COVID-19 infection pending reverse transcriptase polymerase chain reaction (RT-PCR) results, despite recommendations against this. We evaluated radiologist performance for COVID-19 diagnosis on CXR at the time of patient presentation in the Emergency Department (ED).

MATERIALS AND METHODS

We extracted RT-PCR results, clinical history, and CXRs of all patients from a single institution between March and June 2020. 984 RT-PCR positive and 1043 RT-PCR negative radiographs were reviewed by 10 emergency radiologists from 4 academic centers. 100 cases were read by all radiologists and 1927 cases by 2 radiologists. Each radiologist chose the single best label per case: Normal, COVID-19, Other - Infectious, Other - Noninfectious, Non-diagnostic, and Endotracheal Tube. Cases labeled with endotracheal tube (246) or non-diagnostic (54) were excluded. Remaining cases were analyzed for label distribution, clinical history, and inter-reader agreement.

RESULTS

1727 radiographs (732 RT-PCR positive, 995 RT-PCR negative) were included from 1594 patients (51.2% male, 48.8% female, age 59 ± 19 years). For 89 cases read by all readers, there was poor agreement for RT-PCR positive (Fleiss Score 0.36) and negative (Fleiss Score 0.46) exams. Agreement between two readers on 1638 cases was 54.2% (373/688) for RT-PCR positive cases and 71.4% (679/950) for negative cases. Agreement was highest for RT-PCR negative cases labeled as Normal (50.4%, n = 479). Reader performance did not improve with clinical history or time between CXR and RT-PCR result.

CONCLUSION

At the time of presentation to the emergency department, emergency radiologist performance is non-specific for diagnosing COVID-19.

The Predictive Role of Artificial Intelligence-Based Chest CT Quantification in Patients with COVID-19 Pneumonia

We sought to analyze the prognostic value of laboratory and clinical data, and an artificial intelligence (AI)-based algorithm for Coronavirus disease 2019 (COVID-19) severity scoring, on CT-scans of patients hospitalized with COVID-19. Moreover, we aimed to determine personalized probabilities of clinical deterioration. Data of symptomatic patients with COVID-19 who underwent chest-CT-examination at the time of hospital admission between April and November 2020 were analyzed. COVID-19 severity score was automatically quantified for each pulmonary lobe as the percentage of affected lung parenchyma with the AI-based algorithm. Clinical deterioration was defined as a composite of admission to the intensive care unit, need for invasive mechanical ventilation, use of vasopressors or in-hospital mortality. In total 326 consecutive patients were included in the analysis (mean age 66.7 ± 15.3 years, 52.1% male) of whom 85 (26.1%) experienced clinical deterioration. In the multivariable regression analysis prior myocardial infarction (OR = 2.81, 95% CI = 1.12–7.04, p = 0.027), immunodeficiency (OR = 2.08, 95% CI = 1.02–4.25, p = 0.043), C-reactive protein (OR = 1.73, 95% CI = 1.32–2.33, p < 0.001) and AI-based COVID-19 severity score (OR = 1.08; 95% CI = 1.02–1.15, p = 0.013) appeared to be independent predictors of clinical deterioration. Personalized probability values were determined. AI-based COVID-19 severity score assessed at hospital admission can provide additional information about the prognosis of COVID-19, possibly serving as a useful tool for individualized risk-stratification.

Deep learning classification of COVID-19 in chest radiographs: performance and influence of supplemental training

Purpose: Accurate classification of COVID-19 in chest radiographs is invaluable to hard-hit pandemic hot spots. Transfer learning techniques for images using well-known convolutional neural networks show promise in addressing this problem. These methods can significantly benefit from supplemental training on similar conditions, considering that there currently exists no widely available chest x-ray dataset on COVID-19. We evaluate whether targeted pretraining for similar tasks in radiography labeling improves classification performance in a sample radiograph dataset containing COVID-19 cases. Approach: We train a DenseNet121 to classify chest radiographs through six training schemes. Each training scheme is designed to incorporate cases from established datasets for general findings in chest radiography (CXR) and pneumonia, with a control scheme with no pretraining. The resulting six permutations are then trained and evaluated on a dataset of 1060 radiographs collected from 475 patients after March 2020, containing 801 images of laboratory-confirmed COVID-19 cases. Results: Sequential training phases yielded substantial improvement in classification accuracy compared to a baseline of standard transfer learning with ImageNet parameters. The test set area under the receiver operating characteristic curve for COVID-19 classification improved from 0.757 in the control to 0.857 for the optimal training scheme in the available images. Conclusions: We achieve COVID-19 classification accuracies comparable to previous benchmarks of pneumonia classification. Deliberate sequential training, rather than pooling datasets, is critical in training effective COVID-19 classifiers within the limitations of early datasets. These findings bring clinical-grade classification through CXR within reach for more regions impacted by COVID-19.

Development and multicenter validation of chest X-ray radiography interpretations based on natural language processing

Background

Artificial intelligence can assist in interpreting chest X-ray radiography (CXR) data, but large datasets require efficient image annotation. The purpose of this study is to extract CXR labels from diagnostic reports based on natural language processing, train convolutional neural networks (CNNs), and evaluate the classification performance of CNN using CXR data from multiple centers

Methods

We collected the CXR images and corresponding radiology reports of 74,082 subjects as the training dataset. The linguistic entities and relationships from unstructured radiology reports were extracted by the bidirectional encoder representations from transformers (BERT) model, and a knowledge graph was constructed to represent the association between image labels of abnormal signs and the report text of CXR. Then, a 25-label classification system were built to train and test the CNN models with weakly supervised labeling.

Results

In three external test cohorts of 5,996 symptomatic patients, 2,130 screening examinees, and 1,804 community clinic patients, the mean AUC of identifying 25 abnormal signs by CNN reaches 0.866 ± 0.110, 0.891 ± 0.147, and 0.796 ± 0.157, respectively. In symptomatic patients, CNN shows no significant difference with local radiologists in identifying 21 signs (p > 0.05), but is poorer for 4 signs (p < 0.05). In screening examinees, CNN shows no significant difference for 17 signs (p > 0.05), but is poorer at classifying nodules (p = 0.013). In community clinic patients, CNN shows no significant difference for 12 signs (p > 0.05), but performs better for 6 signs (p < 0.001).

Conclusion

We construct and validate an effective CXR interpretation system based on natural language processing.

Chest X-rays are accompanied by a report from the radiologist, which contains valuable diagnostic information in text format. Extracting and interpreting information from these reports, such as keywords, is time-consuming, but artificial intelligence (AI) can help with this. Here, we use a type of AI known as natural language processing to extract information about abnormal signs seen on chest X-rays from the corresponding report. We develop and test natural language processing models using data from multiple hospitals and clinics, and show that our models achieve similar performance to interpretation from the radiologists themselves. Our findings suggest that AI might help radiologists to speed up interpretation of chest X-ray reports, which could be useful not only in patient triage and diagnosis but also cataloguing and searching of radiology datasets.

Zhang et al. develop a natural language processing approach, based on the BERT model, to extract linguistic information from chest X-ray radiography reports. The authors establish a 25-label classification system for abnormal findings described in the reports and validate their model using data from multiple sites.

Artificial Intelligence for COVID-19: A Systematic Review

Background: Recently, Coronavirus Disease 2019 (COVID-19), caused by severe acute respiratory syndrome virus 2 (SARS-CoV-2), has affected more than 200 countries and lead to enormous losses. This study systematically reviews the application of Artificial Intelligence (AI) techniques in COVID-19, especially for diagnosis, estimation of epidemic trends, prognosis, and exploration of effective and safe drugs and vaccines; and discusses the potential limitations. Methods: We report this systematic review following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. We searched PubMed, Embase and the Cochrane Library from inception to 19 September 2020 for published studies of AI applications in COVID-19. We used PROBAST (prediction model risk of bias assessment tool) to assess the quality of literature related to the diagnosis and prognosis of COVID-19. We registered the protocol (PROSPERO CRD42020211555). Results: We included 78 studies: 46 articles discussed AI-assisted diagnosis for COVID-19 with total accuracy of 70.00 to 99.92%, sensitivity of 73.00 to 100.00%, specificity of 25 to 100.00%, and area under the curve of 0.732 to 1.000. Fourteen articles evaluated prognosis based on clinical characteristics at hospital admission, such as clinical, laboratory and radiological characteristics, reaching accuracy of 74.4 to 95.20%, sensitivity of 72.8 to 98.00%, specificity of 55 to 96.87% and AUC of 0.66 to 0.997 in predicting critical COVID-19. Nine articles used AI models to predict the epidemic of the COVID-19, such as epidemic peak, infection rate, number of infected cases, transmission laws, and development trend. Eight articles used AI to explore potential effective drugs, primarily through drug repurposing and drug development. Finally, 1 article predicted vaccine targets that have the potential to develop COVID-19 vaccines. Conclusions: In this review, we have shown that AI achieved high performance in diagnosis, prognosis evaluation, epidemic prediction and drug discovery for COVID-19. AI has the potential to enhance significantly existing medical and healthcare system efficiency during the COVID-19 pandemic.

A Promising and Challenging Approach: Radiologists' Perspective on Deep Learning and Artificial Intelligence for Fighting COVID-19

Chest X-rays (CXR) and computed tomography (CT) are the main medical imaging modalities used against the increased worldwide spread of the 2019 coronavirus disease (COVID-19) epidemic. Machine learning (ML) and artificial intelligence (AI) technology, based on medical imaging fully extracting and utilizing the hidden information in massive medical imaging data, have been used in COVID-19 research of disease diagnosis and classification, treatment decision-making, efficacy evaluation, and prognosis prediction. This review article describes the extensive research of medical image-based ML and AI methods in preventing and controlling COVID-19, and summarizes their characteristics, differences, and significance in terms of application direction, image collection, and algorithm improvement, from the perspective of radiologists. The limitations and challenges faced by these systems and technologies, such as generalization and robustness, are discussed to indicate future research directions.

Development and prospective validation of COVID-19 chest X-ray screening model for patients attending emergency departments

Chest X-rays (CXRs) are the first-line investigation in patients presenting to emergency departments (EDs) with dyspnoea and are a valuable adjunct to clinical management of COVID-19 associated lung disease. Artificial intelligence (AI) has the potential to facilitate rapid triage of CXRs for further patient testing and/or isolation. In this work we develop an AI algorithm, CovIx, to differentiate normal, abnormal, non-COVID-19 pneumonia, and COVID-19 CXRs using a multicentre cohort of 293,143 CXRs. The algorithm is prospectively validated in 3289 CXRs acquired from patients presenting to ED with symptoms of COVID-19 across four sites in NHS Greater Glasgow and Clyde. CovIx achieves area under receiver operating characteristic curve for COVID-19 of 0.86, with sensitivity and F1-score up to 0.83 and 0.71 respectively, and performs on-par with four board-certified radiologists. AI-based algorithms can identify CXRs with COVID-19 associated pneumonia, as well as distinguish non-COVID pneumonias in symptomatic patients presenting to ED. Pre-trained models and inference scripts are freely available at https://github.com/beringresearch/bravecx-covid.

Elaboration of a Radiomics Strategy for the Prediction of the Re-positive Cases in the Discharged Patients With COVID-19

Objective: A considerable part of COVID-19 patients were found to be re-positive in the SARS-CoV-2 RT-PCR test after discharge. Early prediction of re-positive COVID-19 cases is of critical importance in determining the isolation period and developing clinical protocols. Materials and Methods: Ninety-one patients discharged from Wanzhou Three Gorges Central Hospital, Chongqing, China, from February 10, 2020 to March 3, 2020 were administered nasopharyngeal swab SARS-CoV-2 tests within 12-14 days, and 50 eligible patients (32 male and 18 female) with completed data were enrolled. Average age was 48 ± 11.5 years. All patients underwent non-enhanced chest CT on admission. A total of 568 radiomics features were extracted from the CT images, and 17 clinical factors were collected based on the medical record. Student's t-test and support vector machine-based recursive feature elimination (SVM-RFE) method were used to determine an optimal subset of features for the discriminative model development. Results: After Student's t-test, 62 radiomics features showed significant inter-group differences (p < 0.05) between the re-positive and negative cases, and none of the clinical features showed significant differences. These significant features were further selected by SVM-RFE algorithm, and a more compact feature subset containing only two radiomics features was finally determined, achieving the best predictive performance with the accuracy and area under the curve of 72.6% and 0.773 for the identification of the re-positive case. Conclusion: The proposed radiomics method has preliminarily shown potential in identifying the re-positive cases among the recovered COVID-19 patients after discharge. More strategies are to be integrated into the current pipeline to improve its precision, and a larger database with multi-clinical enrollment is required to extensively verify its performance.

Rapidly deploying a COVID-19 decision support system in one of the largest Brazilian hospitals

The COVID-19 pandemic generated research interest in automated models to perform classification and segmentation from medical imaging of COVID-19 patients, However, applications in real-world scenarios are still needed. We describe the development and deployment of COVID-19 decision support and segmentation system. A partnership with a Brazilian radiologist consortium, gave us access to 1000s of labeled computed tomography (CT) and X-ray images from São Paulo Hospitals. The system used EfficientNet and EfficientDet networks, state-of-the-art convolutional neural networks for natural images classification and segmentation, in a real-time scalable scenario in communication with a Picture Archiving and Communication System (PACS). Additionally, the system could reject non-related images, using header analysis and classifiers. We achieved CT and X-ray classification accuracies of 0.94 and 0.98, respectively, and Dice coefficient for lung and covid findings segmentations of 0.98 and 0.73, respectively. The median response time was 7 s for X-ray and 4 min for CT.

Deep convolution neural networks to differentiate between COVID-19 and other pulmonary abnormalities on chest radiographs: Evaluation using internal and external datasets

We aimed to evaluate the performance of convolutional neural networks (CNNs) in the classification of coronavirus disease 2019 (COVID-19) disease using normal, pneumonia, and COVID-19 chest radiographs (CXRs). First, we collected 9194 CXRs from open datasets and 58 from the Korea University Anam Hospital (KUAH). The number of normal, pneumonia, and COVID-19 CXRs were 4580, 3884, and 730, respectively. The CXRs obtained from the open dataset were randomly assigned to the training, tuning, and test sets in a 70:10:20 ratio. For external validation, the KUAH (20 normal, 20 pneumonia, and 18 COVID-19) dataset, verified by radiologists using computed tomography, was used. Subsequently, transfer learning was conducted using DenseNet169, InceptionResNetV2, and Xception to identify COVID-19 using open datasets (internal) and the KUAH dataset (external) with histogram matching. Gradient-weighted class activation mapping was used for the visualization of abnormal patterns in CXRs. The average AUC and accuracy of the multiscale and mixed-COVID-19Net using three CNNs over five folds were (0.99 ± 0.01 and 92.94% ± 0.45%), (0.99 ± 0.01 and 93.12% ± 0.23%), and (0.99 ± 0.01 and 93.57% ± 0.29%), respectively, using the open datasets (internal). Furthermore, these values were (0.75 and 74.14%), (0.72 and 68.97%), and (0.77 and 68.97%), respectively, for the best model among the fivefold cross-validation with the KUAH dataset (external) using domain adaptation. The various state-of-the-art models trained on open datasets show satisfactory performance for clinical interpretation. Furthermore, the domain adaptation for external datasets was found to be important for detecting COVID-19 as well as other diseases.

Medical imaging and computational image analysis in COVID-19 diagnosis: A review

Coronavirus disease (COVID-19) is an infectious disease caused by a newly discovered coronavirus. The disease presents with symptoms such as shortness of breath, fever, dry cough, and chronic fatigue, amongst others. The disease may be asymptomatic in some patients in the early stages, which can lead to increased transmission of the disease to others. This study attempts to review papers on the role of imaging and medical image computing in COVID-19 diagnosis. For this purpose, PubMed, Scopus and Google Scholar were searched to find related studies until the middle of 2021. The contribution of this study is four-fold: 1) to use as a tutorial of the field for both clinicians and technologists, 2) to comprehensively review the characteristics of COVID-19 as presented in medical images, 3) to examine automated artificial intelligence-based approaches for COVID-19 diagnosis, 4) to express the research limitations in this field and the methods used to overcome them. Using machine learning-based methods can diagnose the disease with high accuracy from medical images and reduce time, cost and error of diagnostic procedure. It is recommended to collect bulk imaging data from patients in the shortest possible time to improve the performance of COVID-19 automated diagnostic methods.

The Clinical Information Systems Response to the COVID-19 Pandemic

OBJECTIVE

The year 2020 was predominated by the coronavirus disease 2019 (COVID-19) pandemic. The objective of this article is to review the areas in which clinical information systems (CIS) can be and have been utilized to support and enhance the response of healthcare systems to pandemics, focusing on COVID-19.

METHODS

PubMed/MEDLINE, Google Scholar, the tables of contents of major informatics journals, and the bibliographies of articles were searched for studies pertaining to CIS, pandemics, and COVID-19 through October 2020. The most informative and detailed studies were highlighted, while many others were referenced.

RESULTS

CIS were heavily relied upon by health systems and governmental agencies worldwide in response to COVID-19. Technology-based screening tools were developed to assist rapid case identification and appropriate triaging. Clinical care was supported by utilizing the electronic health record (EHR) to onboard frontline providers to new protocols, offer clinical decision support, and improve systems for diagnostic testing. Telehealth became the most rapidly adopted medical trend in recent history and an essential strategy for allowing safe and effective access to medical care. Artificial intelligence and machine learning algorithms were developed to enhance screening, diagnostic imaging, and predictive analytics - though evidence of improved outcomes remains limited. Geographic information systems and big data enabled real-time dashboards vital for epidemic monitoring, hospital preparedness strategies, and health policy decision making. Digital contact tracing systems were implemented to assist a labor-intensive task with the aim of curbing transmission. Large scale data sharing, effective health information exchange, and interoperability of EHRs remain challenges for the informatics community with immense clinical and academic potential. CIS must be used in combination with engaged stakeholders and operational change management in order to meaningfully improve patient outcomes.

CONCLUSION

Managing a pandemic requires widespread, timely, and effective distribution of reliable information. In the past year, CIS and informaticists made prominent and influential contributions in the global response to the COVID-19 pandemic.

The Applications of Artificial Intelligence in Chest Imaging of COVID-19 Patients: A Literature Review

Diagnostic imaging is regarded as fundamental in the clinical work-up of patients with a suspected or confirmed COVID-19 infection. Recent progress has been made in diagnostic imaging with the integration of artificial intelligence (AI) and machine learning (ML) algorisms leading to an increase in the accuracy of exam interpretation and to the extraction of prognostic information useful in the decision-making process. Considering the ever expanding imaging data generated amid this pandemic, COVID-19 has catalyzed the rapid expansion in the application of AI to combat disease. In this context, many recent studies have explored the role of AI in each of the presumed applications for COVID-19 infection chest imaging, suggesting that implementing AI applications for chest imaging can be a great asset for fast and precise disease screening, identification and characterization. However, various biases should be overcome in the development of further ML-based algorithms to give them sufficient robustness and reproducibility for their integration into clinical practice. As a result, in this literature review, we will focus on the application of AI in chest imaging, in particular, deep learning, radiomics and advanced imaging as quantitative CT.

A comprehensive review of imaging findings in COVID-19 - status in early 2021

Medical imaging methods are assuming a greater role in the workup of patients with COVID-19, mainly in relation to the primary manifestation of pulmonary disease and the tissue distribution of the angiotensin-converting-enzyme 2 (ACE 2) receptor. However, the field is so new that no consensus view has emerged guiding clinical decisions to employ imaging procedures such as radiography, computer tomography (CT), positron emission tomography (PET), and magnetic resonance imaging, and in what measure the risk of exposure of staff to possible infection could be justified by the knowledge gained. The insensitivity of current RT-PCR methods for positive diagnosis is part of the rationale for resorting to imaging procedures. While CT is more sensitive than genetic testing in hospitalized patients, positive findings of ground glass opacities depend on the disease stage. There is sparse reporting on PET/CT with [18F]-FDG in COVID-19, but available results are congruent with the earlier literature on viral pneumonias. There is a high incidence of cerebral findings in COVID-19, and likewise evidence of gastrointestinal involvement. Artificial intelligence, notably machine learning is emerging as an effective method for diagnostic image analysis, with performance in the discriminative diagnosis of diagnosis of COVID-19 pneumonia comparable to that of human practitioners.

COVID-19 pneumonia on chest X-rays: Performance of a deep learning-based computer-aided detection system

Chest X-rays (CXRs) can help triage for Coronavirus disease (COVID-19) patients in resource-constrained environments, and a computer-aided detection system (CAD) that can identify pneumonia on CXR may help the triage of patients in those environment where expert radiologists are not available. However, the performance of existing CAD for identifying COVID-19 and associated pneumonia on CXRs has been scarcely investigated. In this study, CXRs of patients with and without COVID-19 confirmed by reverse transcriptase polymerase chain reaction (RT-PCR) were retrospectively collected from four and one institution, respectively, and a commercialized, regulatory-approved CAD that can identify various abnormalities including pneumonia was used to analyze each CXR. Performance of the CAD was evaluated using area under the receiver operating characteristic curves (AUCs), with reference standards of the RT-PCR results and the presence of findings of pneumonia on chest CTs obtained within 24 hours from the CXR. For comparison, 5 thoracic radiologists and 5 non-radiologist physicians independently interpreted the CXRs. Afterward, they re-interpreted the CXRs with corresponding CAD results. The performance of CAD (AUCs, 0.714 and 0.790 against RT-PCR and chest CT, respectively hereinafter) were similar with those of thoracic radiologists (AUCs, 0.701 and 0.784), and higher than those of non-radiologist physicians (AUCs, 0.584 and 0.650). Non-radiologist physicians showed significantly improved performance when assisted with the CAD (AUCs, 0.584 to 0.664 and 0.650 to 0.738). In addition, inter-reader agreement among physicians was also improved in the CAD-assisted interpretation (Fleiss' kappa coefficient, 0.209 to 0.322). In conclusion, radiologist-level performance of the CAD in identifying COVID-19 and associated pneumonia on CXR and enhanced performance of non-radiologist physicians with the CAD assistance suggest that the CAD can support physicians in interpreting CXRs and helping image-based triage of COVID-19 patients in resource-constrained environment.

Deep learning for chest X-ray analysis: A survey

Recent advances in deep learning have led to a promising performance in many medical image analysis tasks. As the most commonly performed radiological exam, chest radiographs are a particularly important modality for which a variety of applications have been researched. The release of multiple, large, publicly available chest X-ray datasets in recent years has encouraged research interest and boosted the number of publications. In this paper, we review all studies using deep learning on chest radiographs published before March 2021, categorizing works by task: image-level prediction (classification and regression), segmentation, localization, image generation and domain adaptation. Detailed descriptions of all publicly available datasets are included and commercial systems in the field are described. A comprehensive discussion of the current state of the art is provided, including caveats on the use of public datasets, the requirements of clinically useful systems and gaps in the current literature.

Risk Stratification for ECMO Requirement in COVID-19 ICU Patients Using Quantitative Imaging Features in CT Scans on Admission

(1) Background: Extracorporeal membrane oxygenation (ECMO) therapy in intensive care units (ICUs) remains the last treatment option for Coronavirus disease 2019 (COVID-19) patients with severely affected lungs but is highly resource demanding. Early risk stratification for the need of ECMO therapy upon admission to the hospital using artificial intelligence (AI)-based computed tomography (CT) assessment and clinical scores is beneficial for patient assessment and resource management; (2) Methods: Retrospective single-center study with 95 confirmed COVID-19 patients admitted to the participating ICUs. Patients requiring ECMO therapy (n = 14) during ICU stay versus patients without ECMO treatment (n = 81) were evaluated for discriminative clinical prediction parameters and AI-based CT imaging features and their diagnostic potential to predict ECMO therapy. Reported patient data include clinical scores, AI-based CT findings and patient outcomes; (3) Results: Patients subsequently allocated to ECMO therapy had significantly higher sequential organ failure (SOFA) scores (p < 0.001) and significantly lower oxygenation indices on admission (p = 0.009) than patients with standard ICU therapy. The median time from hospital admission to ECMO placement was 1.4 days (IQR 0.2-4.0). The percentage of lung involvement on AI-based CT assessment on admission to the hospital was significantly higher in ECMO patients (p < 0.001). In binary logistic regression analyses for ECMO prediction including age, sex, body mass index (BMI), SOFA score on admission, lactate on admission and percentage of lung involvement on admission CTs, only SOFA score (OR 1.32, 95% CI 1.08-1.62) and lung involvement (OR 1.06, 95% CI 1.01-1.11) were significantly associated with subsequent ECMO allocation. Receiver operating characteristic (ROC) curves showed an area under the curve (AUC) of 0.83 (95% CI 0.73-0.94) for lung involvement on admission CT and 0.82 (95% CI 0.72-0.91) for SOFA scores on ICU admission. A combined parameter of SOFA on ICU admission and lung involvement on admission CT yielded an AUC of 0.91 (0.84-0.97) with a sensitivity of 0.93 and a specificity of 0.84 for ECMO prediction; (4) Conclusions: AI-based assessment of lung involvement on CT scans on admission to the hospital and SOFA scoring, especially if combined, can be used as risk stratification tools for subsequent requirement for ECMO therapy in patients with severe COVID-19 disease to improve resource management in ICU settings.

A Deep-Learning Diagnostic Support System for the Detection of COVID-19 Using Chest Radiographs: A Multireader Validation Study

MATERIALS AND METHODS

Five publicly available databases comprising normal CXR, confirmed COVID-19 pneumonia cases, and other pneumonias were used. After the harmonization of the data, the training set included 7966 normal cases, 5451 with other pneumonia, and 258 CXRs with COVID-19 pneumonia, whereas in the testing data set, each category was represented by 100 cases. Eleven blinded radiologists with various levels of expertise independently read the testing data set. The data were analyzed separately with the newly proposed artificial intelligence-based system and by consultant radiologists and residents, with respect to positive predictive value (PPV), sensitivity, and F-score (harmonic mean for PPV and sensitivity). The χ2 test was used to compare the sensitivity, specificity, accuracy, PPV, and F-scores of the readers and the system.

RESULTS

The proposed system achieved higher overall diagnostic accuracy (94.3%) than the radiologists (61.4% ± 5.3%). The radiologists reached average sensitivities for normal CXR, other type of pneumonia, and COVID-19 pneumonia of 85.0% ± 12.8%, 60.1% ± 12.2%, and 53.2% ± 11.2%, respectively, which were significantly lower than the results achieved by the algorithm (98.0%, 88.0%, and 97.0%; P < 0.00032). The mean PPVs for all 11 radiologists for the 3 categories were 82.4%, 59.0%, and 59.0% for the healthy, other pneumonia, and COVID-19 pneumonia, respectively, resulting in an F-score of 65.5% ± 12.4%, which was significantly lower than the F-score of the algorithm (94.3% ± 2.0%, P < 0.00001). When other pneumonia and COVID-19 pneumonia cases were pooled, the proposed system reached an accuracy of 95.7% for any pathology and the radiologists, 88.8%. The overall accuracy of consultants did not vary significantly compared with residents (65.0% ± 5.8% vs 67.4% ± 4.2%); however, consultants detected significantly more COVID-19 pneumonia cases (P = 0.008) and less healthy cases (P < 0.00001).

CONCLUSIONS

The system showed robust accuracy for COVID-19 pneumonia detection on CXR and surpassed radiologists at various training levels.

An automated and fast system to identify COVID-19 from X-ray radiograph of the chest using image processing and machine learning

A type of coronavirus disease called COVID-19 is spreading all over the globe. Researchers and scientists are endeavoring to find new and effective methods to diagnose and treat this disease. This article presents an automated and fast system that identifies COVID-19 from X-ray radiographs of the chest using image processing and machine learning algorithms. Initially, the system extracts the feature descriptors from the radiographs of both healthy and COVID-19 affected patients using the speeded up robust features algorithm. Then, visual vocabulary is built by reducing the number of feature descriptors via quantization of feature space using the K-means clustering algorithm. The visual vocabulary train the support vector machine (SVM) classifier. During testing, an X-ray radiograph's visual vocabulary is sent to the trained SVM classifier to detect the absence or presence of COVID-19. The study used the dataset of 340 X-ray radiographs, 170 images of each Healthy and Positive COVID-19 class. During simulations, the dataset split into training and testing parts at various ratios. After training, the system does not require any human intervention and can process thousands of images with high precision in a few minutes. The performance of the system is measured using standard parameters of accuracy and confusion matrix. We compared the performance of the proposed SVM-based classier with the deep-learning-based convolutional neural networks (CNN). The SVM yields better results than CNN and achieves a maximum accuracy of up to 94.12%.

COVID-19 Imaging: What We Know Now and What Remains Unknown

Infection with SARS-CoV-2 ranges from an asymptomatic condition to a severe and sometimes fatal disease, with mortality most frequently being the result of acute lung injury. The role of imaging has evolved during the pandemic, with CT initially being an alternative and possibly superior testing method compared with reverse transcriptase-polymerase chain reaction (RT-PCR) testing and evolving to having a more limited role based on specific indications. Several classification and reporting schemes were developed for chest imaging early during the pandemic for patients suspected of having COVID-19 to aid in triage when the availability of RT-PCR testing was limited and its level of performance was unclear. Interobserver agreement for categories with findings typical of COVID-19 and those suggesting an alternative diagnosis is high across multiple studies. Furthermore, some studies looking at the extent of lung involvement on chest radiographs and CT images showed correlations with critical illness and a need for mechanical ventilation. In addition to pulmonary manifestations, cardiovascular complications such as thromboembolism and myocarditis have been ascribed to COVID-19, sometimes contributing to neurologic and abdominal manifestations. Finally, artificial intelligence has shown promise for use in determining both the diagnosis and prognosis of COVID-19 pneumonia with respect to both radiography and CT.

Detection of COVID-19 Using Deep Learning Algorithms on Chest Radiographs

PURPOSE

To evaluate the performance of a deep learning (DL) algorithm for the detection of COVID-19 on chest radiographs (CXR).

MATERIALS AND METHODS

In this retrospective study, a DL model was trained on 112,120 CXR images with 14 labeled classifiers (ChestX-ray14) and fine-tuned using initial CXR on hospital admission of 509 patients, who had undergone COVID-19 reverse transcriptase-polymerase chain reaction (RT-PCR). The test set consisted of a CXR on presentation of 248 individuals suspected of COVID-19 pneumonia between February 16 and March 3, 2020 from 4 centers (72 RT-PCR positives and 176 RT-PCR negatives). The CXR were independently reviewed by 3 radiologists and using the DL algorithm. Diagnostic performance was compared with radiologists' performance and was assessed by area under the receiver operating characteristics (AUC).

RESULTS

The median age of the subjects in the test set was 61 (interquartile range: 39 to 79) years (51% male). The DL algorithm achieved an AUC of 0.81, sensitivity of 0.85, and specificity of 0.72 in detecting COVID-19 using RT-PCR as the reference standard. On subgroup analyses, the model achieved an AUC of 0.79, sensitivity of 0.80, and specificity of 0.74 in detecting COVID-19 in patients presented with fever or respiratory systems and an AUC of 0.87, sensitivity of 0.85, and specificity of 0.81 in distinguishing COVID-19 from other forms of pneumonia. The algorithm significantly outperforms human readers (P<0.001 using DeLong test) with higher sensitivity (P=0.01 using McNemar test).

CONCLUSIONS

A DL algorithm (COV19NET) for the detection of COVID-19 on chest radiographs can potentially be an effective tool in triaging patients, particularly in resource-stretched health-care systems.

A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises

Since its renaissance, deep learning has been widely used in various medical imaging tasks and has achieved remarkable success in many medical imaging applications, thereby propelling us into the so-called artificial intelligence (AI) era. It is known that the success of AI is mostly attributed to the availability of big data with annotations for a single task and the advances in high performance computing. However, medical imaging presents unique challenges that confront deep learning approaches. In this survey paper, we first present traits of medical imaging, highlight both clinical needs and technical challenges in medical imaging, and describe how emerging trends in deep learning are addressing these issues. We cover the topics of network architecture, sparse and noisy labels, federating learning, interpretability, uncertainty quantification, etc. Then, we present several case studies that are commonly found in clinical practice, including digital pathology and chest, brain, cardiovascular, and abdominal imaging. Rather than presenting an exhaustive literature survey, we instead describe some prominent research highlights related to these case study applications. We conclude with a discussion and presentation of promising future directions.

The Role of Artificial Intelligence in Fighting the COVID-19 Pandemic

The first few months of 2020 have profoundly changed the way we live our lives and carry out our daily activities. Although the widespread use of futuristic robotaxis and self-driving commercial vehicles has not yet become a reality, the COVID-19 pandemic has dramatically accelerated the adoption of Artificial Intelligence (AI) in different fields. We have witnessed the equivalent of two years of digital transformation compressed into just a few months. Whether it is in tracing epidemiological peaks or in transacting contactless payments, the impact of these developments has been almost immediate, and a window has opened up on what is to come. Here we analyze and discuss how AI can support us in facing the ongoing pandemic. Despite the numerous and undeniable contributions of AI, clinical trials and human skills are still required. Even if different strategies have been developed in different states worldwide, the fight against the pandemic seems to have found everywhere a valuable ally in AI, a global and open-source tool capable of providing assistance in this health emergency. A careful AI application would enable us to operate within this complex scenario involving healthcare, society and research.

Machine Learning Models for Image-Based Diagnosis and Prognosis of COVID-19: Systematic Review

BACKGROUND

Accurate and timely diagnosis and effective prognosis of the disease is important to provide the best possible care for patients with COVID-19 and reduce the burden on the health care system. Machine learning methods can play a vital role in the diagnosis of COVID-19 by processing chest x-ray images.

OBJECTIVE

The aim of this study is to summarize information on the use of intelligent models for the diagnosis and prognosis of COVID-19 to help with early and timely diagnosis, minimize prolonged diagnosis, and improve overall health care.

METHODS

A systematic search of databases, including PubMed, Web of Science, IEEE, ProQuest, Scopus, bioRxiv, and medRxiv, was performed for COVID-19-related studies published up to May 24, 2020. This study was performed in accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) guidelines. All original research articles describing the application of image processing for the prediction and diagnosis of COVID-19 were considered in the analysis. Two reviewers independently assessed the published papers to determine eligibility for inclusion in the analysis. Risk of bias was evaluated using the Prediction Model Risk of Bias Assessment Tool.

RESULTS

Of the 629 articles retrieved, 44 articles were included. We identified 4 prognosis models for calculating prediction of disease severity and estimation of confinement time for individual patients, and 40 diagnostic models for detecting COVID-19 from normal or other pneumonias. Most included studies used deep learning methods based on convolutional neural networks, which have been widely used as a classification algorithm. The most frequently reported predictors of prognosis in patients with COVID-19 included age, computed tomography data, gender, comorbidities, symptoms, and laboratory findings. Deep convolutional neural networks obtained better results compared with non-neural network-based methods. Moreover, all of the models were found to be at high risk of bias due to the lack of information about the study population, intended groups, and inappropriate reporting.

CONCLUSIONS

Machine learning models used for the diagnosis and prognosis of COVID-19 showed excellent discriminative performance. However, these models were at high risk of bias, because of various reasons such as inadequate information about study participants, randomization process, and the lack of external validation, which may have resulted in the optimistic reporting of these models. Hence, our findings do not recommend any of the current models to be used in practice for the diagnosis and prognosis of COVID-19.

DeepCOVID-XR: An Artificial Intelligence Algorithm to Detect COVID-19 on Chest Radiographs Trained and Tested on a Large U.S. Clinical Data Set

Background There are characteristic findings of coronavirus disease 2019 (COVID-19) on chest images. An artificial intelligence (AI) algorithm to detect COVID-19 on chest radiographs might be useful for triage or infection control within a hospital setting, but prior reports have been limited by small data sets, poor data quality, or both. Purpose To present DeepCOVID-XR, a deep learning AI algorithm to detect COVID-19 on chest radiographs, that was trained and tested on a large clinical data set. Materials and Methods DeepCOVID-XR is an ensemble of convolutional neural networks developed to detect COVID-19 on frontal chest radiographs, with reverse-transcription polymerase chain reaction test results as the reference standard. The algorithm was trained and validated on 14 788 images (4253 positive for COVID-19) from sites across the Northwestern Memorial Health Care System from February 2020 to April 2020 and was then tested on 2214 images (1192 positive for COVID-19) from a single hold-out institution. Performance of the algorithm was compared with interpretations from five experienced thoracic radiologists on 300 random test images using the McNemar test for sensitivity and specificity and the DeLong test for the area under the receiver operating characteristic curve (AUC). Results A total of 5853 patients (mean age, 58 years ± 19 [standard deviation]; 3101 women) were evaluated across data sets. For the entire test set, accuracy of DeepCOVID-XR was 83%, with an AUC of 0.90. For 300 random test images (134 positive for COVID-19), accuracy of DeepCOVID-XR was 82%, compared with that of individual radiologists (range, 76%-81%) and the consensus of all five radiologists (81%). DeepCOVID-XR had a significantly higher sensitivity (71%) than one radiologist (60%, P < .001) and significantly higher specificity (92%) than two radiologists (75%, P < .001; 84%, P = .009). AUC of DeepCOVID-XR was 0.88 compared with the consensus AUC of 0.85 (P = .13 for comparison). With consensus interpretation as the reference standard, the AUC of DeepCOVID-XR was 0.95 (95% CI: 0.92, 0.98). Conclusion DeepCOVID-XR, an artificial intelligence algorithm, detected coronavirus disease 2019 on chest radiographs with a performance similar to that of experienced thoracic radiologists in consensus. © RSNA, 2020 Supplemental material is available for this article. See also the editorial by van Ginneken in this issue.

Thoracic imaging tests for the diagnosis of COVID-19

BACKGROUND

The respiratory illness caused by SARS-CoV-2 infection continues to present diagnostic challenges. Our 2020 edition of this review showed thoracic (chest) imaging to be sensitive and moderately specific in the diagnosis of coronavirus disease 2019 (COVID-19). In this update, we include new relevant studies, and have removed studies with case-control designs, and those not intended to be diagnostic test accuracy studies.

OBJECTIVES

To evaluate the diagnostic accuracy of thoracic imaging (computed tomography (CT), X-ray and ultrasound) in people with suspected COVID-19.

SEARCH METHODS

We searched the COVID-19 Living Evidence Database from the University of Bern, the Cochrane COVID-19 Study Register, The Stephen B. Thacker CDC Library, and repositories of COVID-19 publications through to 30 September 2020. We did not apply any language restrictions.

SELECTION CRITERIA

We included studies of all designs, except for case-control, that recruited participants of any age group suspected to have COVID-19 and that reported estimates of test accuracy or provided data from which we could compute estimates.

DATA COLLECTION AND ANALYSIS

The review authors independently and in duplicate screened articles, extracted data and assessed risk of bias and applicability concerns using the QUADAS-2 domain-list. We presented the results of estimated sensitivity and specificity using paired forest plots, and we summarised pooled estimates in tables. We used a bivariate meta-analysis model where appropriate. We presented the uncertainty of accuracy estimates using 95% confidence intervals (CIs).

MAIN RESULTS

We included 51 studies with 19,775 participants suspected of having COVID-19, of whom 10,155 (51%) had a final diagnosis of COVID-19. Forty-seven studies evaluated one imaging modality each, and four studies evaluated two imaging modalities each. All studies used RT-PCR as the reference standard for the diagnosis of COVID-19, with 47 studies using only RT-PCR and four studies using a combination of RT-PCR and other criteria (such as clinical signs, imaging tests, positive contacts, and follow-up phone calls) as the reference standard. Studies were conducted in Europe (33), Asia (13), North America (3) and South America (2); including only adults (26), all ages (21), children only (1), adults over 70 years (1), and unclear (2); in inpatients (2), outpatients (32), and setting unclear (17). Risk of bias was high or unclear in thirty-two (63%) studies with respect to participant selection, 40 (78%) studies with respect to reference standard, 30 (59%) studies with respect to index test, and 24 (47%) studies with respect to participant flow. For chest CT (41 studies, 16,133 participants, 8110 (50%) cases), the sensitivity ranged from 56.3% to 100%, and specificity ranged from 25.4% to 97.4%. The pooled sensitivity of chest CT was 87.9% (95% CI 84.6 to 90.6) and the pooled specificity was 80.0% (95% CI 74.9 to 84.3). There was no statistical evidence indicating that reference standard conduct and definition for index test positivity were sources of heterogeneity for CT studies. Nine chest CT studies (2807 participants, 1139 (41%) cases) used the COVID-19 Reporting and Data System (CO-RADS) scoring system, which has five thresholds to define index test positivity. At a CO-RADS threshold of 5 (7 studies), the sensitivity ranged from 41.5% to 77.9% and the pooled sensitivity was 67.0% (95% CI 56.4 to 76.2); the specificity ranged from 83.5% to 96.2%; and the pooled specificity was 91.3% (95% CI 87.6 to 94.0). At a CO-RADS threshold of 4 (7 studies), the sensitivity ranged from 56.3% to 92.9% and the pooled sensitivity was 83.5% (95% CI 74.4 to 89.7); the specificity ranged from 77.2% to 90.4% and the pooled specificity was 83.6% (95% CI 80.5 to 86.4). For chest X-ray (9 studies, 3694 participants, 2111 (57%) cases) the sensitivity ranged from 51.9% to 94.4% and specificity ranged from 40.4% to 88.9%. The pooled sensitivity of chest X-ray was 80.6% (95% CI 69.1 to 88.6) and the pooled specificity was 71.5% (95% CI 59.8 to 80.8). For ultrasound of the lungs (5 studies, 446 participants, 211 (47%) cases) the sensitivity ranged from 68.2% to 96.8% and specificity ranged from 21.3% to 78.9%. The pooled sensitivity of ultrasound was 86.4% (95% CI 72.7 to 93.9) and the pooled specificity was 54.6% (95% CI 35.3 to 72.6). Based on an indirect comparison using all included studies, chest CT had a higher specificity than ultrasound. For indirect comparisons of chest CT and chest X-ray, or chest X-ray and ultrasound, the data did not show differences in specificity or sensitivity.

AUTHORS' CONCLUSIONS

Our findings indicate that chest CT is sensitive and moderately specific for the diagnosis of COVID-19. Chest X-ray is moderately sensitive and moderately specific for the diagnosis of COVID-19. Ultrasound is sensitive but not specific for the diagnosis of COVID-19. Thus, chest CT and ultrasound may have more utility for excluding COVID-19 than for differentiating SARS-CoV-2 infection from other causes of respiratory illness. Future diagnostic accuracy studies should pre-define positive imaging findings, include direct comparisons of the various modalities of interest in the same participant population, and implement improved reporting practices.

Lessons from the COVID-19 Pandemic on the Use of Artificial Intelligence in Digital Radiology: The Submission of a Survey to Investigate the Opinion of Insiders

The development of artificial intelligence (AI) during the COVID-19 pandemic is there for all to see, and has undoubtedly mainly concerned the activities of digital radiology. Nevertheless, the strong perception in the research and clinical application environment is that AI in radiology is like a hammer in search of a nail. Notable developments and opportunities do not seem to be combined, now, in the time of the COVID-19 pandemic, with a stable, effective, and concrete use in clinical routine; the use of AI often seems limited to use in research applications. This study considers the future perceived integration of AI with digital radiology after the COVID-19 pandemic and proposes a methodology that, by means of a wide interaction of the involved actors, allows a positioning exercise for acceptance evaluation using a general purpose electronic survey. The methodology was tested on a first category of professionals, the medical radiology technicians (MRT), and allowed to (i) collect their impressions on the issue in a structured way, and (ii) collect their suggestions and their comments in order to create a specific tool for this professional figure to be used in scientific societies. This study is useful for the stakeholders in the field, and yielded several noteworthy observations, among them (iii) the perception of great development in thoracic radiography and CT, but a loss of opportunity in integration with non-radiological technologies; (iv) the belief that it is appropriate to invest in training and infrastructure dedicated to AI; and (v) the widespread idea that AI can become a strong complementary tool to human activity. From a general point of view, the study is a clear invitation to face the last yard of AI in digital radiology, a last yard that depends a lot on the opinion and the ability to accept these technologies by the operators of digital radiology.

A Pictorial Review of the Role of Imaging in the Detection, Management, Histopathological Correlations, and Complications of COVID-19 Pneumonia

Imaging plays an important role in the detection of coronavirus (COVID-19) pneumonia in both managing the disease and evaluating the complications. Imaging with chest computed tomography (CT) can also have a potential predictive and prognostic role in COVID-19 patient outcomes. The aim of this pictorial review is to describe the role of imaging with chest X-ray (CXR), lung ultrasound (LUS), and CT in the diagnosis and management of COVID-19 pneumonia, the current indications, the scores proposed for each modality, the advantages/limitations of each modality and their role in detecting complications, and the histopathological correlations.

Viral Pneumonia Screening on Chest X-Rays Using Confidence-Aware Anomaly Detection

Clusters of viral pneumonia occurrences over a short period may be a harbinger of an outbreak or pandemic. Rapid and accurate detection of viral pneumonia using chest X-rays can be of significant value for large-scale screening and epidemic prevention, particularly when other more sophisticated imaging modalities are not readily accessible. However, the emergence of novel mutated viruses causes a substantial dataset shift, which can greatly limit the performance of classification-based approaches. In this paper, we formulate the task of differentiating viral pneumonia from non-viral pneumonia and healthy controls into a one-class classification-based anomaly detection problem. We therefore propose the confidence-aware anomaly detection (CAAD) model, which consists of a shared feature extractor, an anomaly detection module, and a confidence prediction module. If the anomaly score produced by the anomaly detection module is large enough, or the confidence score estimated by the confidence prediction module is small enough, the input will be accepted as an anomaly case (i.e., viral pneumonia). The major advantage of our approach over binary classification is that we avoid modeling individual viral pneumonia classes explicitly and treat all known viral pneumonia cases as anomalies to improve the one-class model. The proposed model outperforms binary classification models on the clinical X-VIRAL dataset that contains 5,977 viral pneumonia (no COVID-19) cases, 37,393 non-viral pneumonia or healthy cases. Moreover, when directly testing on the X-COVID dataset that contains 106 COVID-19 cases and 107 normal controls without any fine-tuning, our model achieves an AUC of 83.61% and sensitivity of 71.70%, which is comparable to the performance of radiologists reported in the literature.

Combining Initial Radiographs and Clinical Variables Improves Deep Learning Prognostication in Patients with COVID-19 from the Emergency Department

PURPOSE

To train a deep learning classification algorithm to predict chest radiograph severity scores and clinical outcomes in patients with coronavirus disease 2019 (COVID-19).

MATERIALS AND METHODS

In this retrospective cohort study, patients aged 21-50 years who presented to the emergency department (ED) of a multicenter urban health system from March 10 to 26, 2020, with COVID-19 confirmation at real-time reverse-transcription polymerase chain reaction screening were identified. The initial chest radiographs, clinical variables, and outcomes, including admission, intubation, and survival, were collected within 30 days (n = 338; median age, 39 years; 210 men). Two fellowship-trained cardiothoracic radiologists examined chest radiographs for opacities and assigned a clinically validated severity score. A deep learning algorithm was trained to predict outcomes on a holdout test set composed of patients with confirmed COVID-19 who presented between March 27 and 29, 2020 (n = 161; median age, 60 years; 98 men) for both younger (age range, 21-50 years; n = 51) and older (age >50 years, n = 110) populations. Bootstrapping was used to compute CIs.

RESULTS

The model trained on the chest radiograph severity score produced the following areas under the receiver operating characteristic curves (AUCs): 0.80 (95% CI: 0.73, 0.88) for the chest radiograph severity score, 0.76 (95% CI: 0.68, 0.84) for admission, 0.66 (95% CI: 0.56, 0.75) for intubation, and 0.59 (95% CI: 0.49, 0.69) for death. The model trained on clinical variables produced an AUC of 0.64 (95% CI: 0.55, 0.73) for intubation and an AUC of 0.59 (95% CI: 0.50, 0.68) for death. Combining chest radiography and clinical variables increased the AUC of intubation and death to 0.88 (95% CI: 0.79, 0.96) and 0.82 (95% CI: 0.72, 0.91), respectively.

CONCLUSION

The combination of imaging and clinical information improves outcome predictions.Supplemental material is available for this article.© RSNA, 2020.

Initial chest radiographs and artificial intelligence (AI) predict clinical outcomes in COVID-19 patients: analysis of 697 Italian patients

OBJECTIVE

To evaluate whether the initial chest X-ray (CXR) severity assessed by an AI system may have prognostic utility in patients with COVID-19.

METHODS

This retrospective single-center study included adult patients presenting to the emergency department (ED) between February 25 and April 9, 2020, with SARS-CoV-2 infection confirmed on real-time reverse transcriptase polymerase chain reaction (RT-PCR). Initial CXRs obtained on ED presentation were evaluated by a deep learning artificial intelligence (AI) system and compared with the Radiographic Assessment of Lung Edema (RALE) score, calculated by two experienced radiologists. Death and critical COVID-19 (admission to intensive care unit (ICU) or deaths occurring before ICU admission) were identified as clinical outcomes. Independent predictors of adverse outcomes were evaluated by multivariate analyses.

RESULTS

Six hundred ninety-seven 697 patients were included in the study: 465 males (66.7%), median age of 62 years (IQR 52-75). Multivariate analyses adjusting for demographics and comorbidities showed that an AI system-based score ≥ 30 on the initial CXR was an independent predictor both for mortality (HR 2.60 (95% CI 1.69 - 3.99; p < 0.001)) and critical COVID-19 (HR 3.40 (95% CI 2.35-4.94; p < 0.001)). Other independent predictors were RALE score, older age, male sex, coronary artery disease, COPD, and neurodegenerative disease.

CONCLUSION

AI- and radiologist-assessed disease severity scores on CXRs obtained on ED presentation were independent and comparable predictors of adverse outcomes in patients with COVID-19.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04318366 ( https://clinicaltrials.gov/ct2/show/NCT04318366 ).

KEY POINTS

• AI system-based score ≥ 30 and a RALE score ≥ 12 at CXRs performed at ED presentation are independent and comparable predictors of death and/or ICU admission in COVID-19 patients. • Other independent predictors are older age, male sex, coronary artery disease, COPD, and neurodegenerative disease. • The comparable performance of the AI system in relation to a radiologist-assessed score in predicting adverse outcomes may represent a game-changer in resource-constrained settings.

A 53-Year-Old Man Presents to the ED With Shortness of Breath, Cough, and Fever

A 53-year-old man presented to the ED at a time of low severe acute respiratory syndrome coronavirus 2, also known as coronavirus disease 2019 (COVID-19), prevalence and reported 2 weeks of progressive shortness of breath, dry cough, headache, myalgias, diarrhea, and recurrent low-grade fevers to 39°C for 1 week with several days of recorded peripheral capillary oxygen saturation of 80% to 90% (room air) on home pulse oximeter. Five days earlier, he had visited an urgent care center where a routine respiratory viral panel was reportedly negative. A COVID-19 reverse transcriptase polymerase chain reaction test result was pending at the time of ED visit. He reported a past medical history of gastroesophageal reflux disease that was treated with famotidine. Travel history included an out-of-state trip 3 weeks earlier, but no recent international travel.

Automatic classification of canine thoracic radiographs using deep learning

The interpretation of thoracic radiographs is a challenging and error-prone task for veterinarians. Despite recent advancements in machine learning and computer vision, the development of computer-aided diagnostic systems for radiographs remains a challenging and unsolved problem, particularly in the context of veterinary medicine. In this study, a novel method, based on multi-label deep convolutional neural network (CNN), for the classification of thoracic radiographs in dogs was developed. All the thoracic radiographs of dogs performed between 2010 and 2020 in the institution were retrospectively collected. Radiographs were taken with two different radiograph acquisition systems and were divided into two data sets accordingly. One data set (Data Set 1) was used for training and testing and another data set (Data Set 2) was used to test the generalization ability of the CNNs. Radiographic findings used as non mutually exclusive labels to train the CNNs were: unremarkable, cardiomegaly, alveolar pattern, bronchial pattern, interstitial pattern, mass, pleural effusion, pneumothorax, and megaesophagus. Two different CNNs, based on ResNet-50 and DenseNet-121 architectures respectively, were developed and tested. The CNN based on ResNet-50 had an Area Under the Receive-Operator Curve (AUC) above 0.8 for all the included radiographic findings except for bronchial and interstitial patterns both on Data Set 1 and Data Set 2. The CNN based on DenseNet-121 had a lower overall performance. Statistically significant differences in the generalization ability between the two CNNs were evident, with the CNN based on ResNet-50 showing better performance for alveolar pattern, interstitial pattern, megaesophagus, and pneumothorax.

Diagnosis of Coronavirus Disease 2019 Pneumonia by Using Chest Radiography: Value of Artificial Intelligence

Background Radiologists are proficient in differentiating between chest radiographs with and without symptoms of pneumonia but have found it more challenging to differentiate coronavirus disease 2019 (COVID-19) pneumonia from non-COVID-19 pneumonia on chest radiographs. Purpose To develop an artificial intelligence algorithm to differentiate COVID-19 pneumonia from other causes of abnormalities at chest radiography. Materials and Methods In this retrospective study, a deep neural network, CV19-Net, was trained, validated, and tested on chest radiographs in patients with and without COVID-19 pneumonia. For the chest radiographs positive for COVID-19, patients with reverse transcription polymerase chain reaction results positive for severe acute respiratory syndrome coronavirus 2 with findings positive for pneumonia between February 1, 2020, and May 30, 2020, were included. For the non-COVID-19 chest radiographs, patients with pneumonia who underwent chest radiography between October 1, 2019, and December 31, 2019, were included. Area under the receiver operating characteristic curve (AUC), sensitivity, and specificity were calculated to characterize diagnostic performance. To benchmark the performance of CV19-Net, a randomly sampled test data set composed of 500 chest radiographs in 500 patients was evaluated by the CV19-Net and three experienced thoracic radiologists. Results A total of 2060 patients (5806 chest radiographs; mean age, 62 years ± 16 [standard deviation]; 1059 men) with COVID-19 pneumonia and 3148 patients (5300 chest radiographs; mean age, 64 years ± 18; 1578 men) with non-COVID-19 pneumonia were included and split into training and validation and test data sets. For the test set, CV19-Net achieved an AUC of 0.92 (95% CI: 0.91, 0.93). This corresponded to a sensitivity of 88% (95% CI: 87, 89) and a specificity of 79% (95% CI: 77, 80) by using a high-sensitivity operating threshold, or a sensitivity of 78% (95% CI: 77, 79) and a specificity of 89% (95% CI: 88, 90) by using a high-specificity operating threshold. For the 500 sampled chest radiographs, CV19-Net achieved an AUC of 0.94 (95% CI: 0.93, 0.96) compared with an AUC of 0.85 (95% CI: 0.81, 0.88) achieved by radiologists. Conclusion CV19-Net was able to differentiate coronavirus disease 2019-related pneumonia from other types of pneumonia, with performance exceeding that of experienced thoracic radiologists. © RSNA, 2021 Online supplemental material is available for this article.

Artificial Intelligence-assisted chest X-ray assessment scheme for COVID-19

Objectives

To study whether a trained convolutional neural network (CNN) can be of assistance to radiologists in differentiating Coronavirus disease (COVID)–positive from COVID-negative patients using chest X-ray (CXR) through an ambispective clinical study. To identify subgroups of patients where artificial intelligence (AI) can be of particular value and analyse what imaging features may have contributed to the performance of AI by means of visualisation techniques.

Methods

CXR of 487 patients were classified into [4] categories—normal, classical COVID, indeterminate, and non-COVID by consensus opinion of 2 radiologists. CXR which were classified as “normal” and “indeterminate” were then subjected to analysis by AI, and final categorisation provided as guided by prediction of the network. Precision and recall of the radiologist alone and radiologist assisted by AI were calculated in comparison to reverse transcriptase-polymerase chain reaction (RT-PCR) as the gold standard. Attention maps of the CNN were analysed to understand regions in the CXR important to the AI algorithm in making a prediction.

Results

The precision of radiologists improved from 65.9 to 81.9% and recall improved from 17.5 to 71.75 when assistance with AI was provided. AI showed 92% accuracy in classifying “normal” CXR into COVID or non-COVID. Analysis of attention maps revealed attention on the cardiac shadow in these “normal” radiographs.

Conclusion

This study shows how deployment of an AI algorithm can complement a human expert in the determination of COVID status. Analysis of the detected features suggests possible subtle cardiac changes, laying ground for further investigative studies into possible cardiac changes.

Key Points

• Through an ambispective clinical study, we show how assistance with an AI algorithm can improve recall (sensitivity) and precision (positive predictive value) of radiologists in assessing CXR for possible COVID in comparison to RT-PCR.

• We show that AI achieves the best results in images classified as “normal” by radiologists. We conjecture that possible subtle cardiac in the CXR, imperceptible to the human eye, may have contributed to this prediction.

• The reported results may pave the way for a human computer collaboration whereby the expert with some help from the AI algorithm achieves higher accuracy in predicting COVID status on CXR than previously thought possible when considering either alone.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00330-020-07628-5.

Artificial intelligence matches subjective severity assessment of pneumonia for prediction of patient outcome and need for mechanical ventilation: a cohort study

To compare the performance of artificial intelligence (AI) and Radiographic Assessment of Lung Edema (RALE) scores from frontal chest radiographs (CXRs) for predicting patient outcomes and the need for mechanical ventilation in COVID-19 pneumonia. Our IRB-approved study included 1367 serial CXRs from 405 adult patients (mean age 65 ± 16 years) from two sites in the US (Site A) and South Korea (Site B). We recorded information pertaining to patient demographics (age, gender), smoking history, comorbid conditions (such as cancer, cardiovascular and other diseases), vital signs (temperature, oxygen saturation), and available laboratory data (such as WBC count and CRP). Two thoracic radiologists performed the qualitative assessment of all CXRs based on the RALE score for assessing the severity of lung involvement. All CXRs were processed with a commercial AI algorithm to obtain the percentage of the lung affected with findings related to COVID-19 (AI score). Independent t- and chi-square tests were used in addition to multiple logistic regression with Area Under the Curve (AUC) as output for predicting disease outcome and the need for mechanical ventilation. The RALE and AI scores had a strong positive correlation in CXRs from each site (r2 = 0.79-0.86; p < 0.0001). Patients who died or received mechanical ventilation had significantly higher RALE and AI scores than those with recovery or without the need for mechanical ventilation (p < 0.001). Patients with a more substantial difference in baseline and maximum RALE scores and AI scores had a higher prevalence of death and mechanical ventilation (p < 0.001). The addition of patients' age, gender, WBC count, and peripheral oxygen saturation increased the outcome prediction from 0.87 to 0.94 (95% CI 0.90-0.97) for RALE scores and from 0.82 to 0.91 (95% CI 0.87-0.95) for the AI scores. AI algorithm is as robust a predictor of adverse patient outcome (death or need for mechanical ventilation) as subjective RALE scores in patients with COVID-19 pneumonia.

Texture Analysis in the Evaluation of Covid-19 Pneumonia in Chest X-Ray Images: a Proof of Concept Study

BACKGROUND

One of the most challenging aspects related to Covid-19 is to establish the presence of infection in early phase of the disease. Texture analysis might be an additional tool for the evaluation of Chest X-ray in patients with clinical suspicion of Covid-19 related pneumonia.

OBJECTIVE

To evaluate the diagnostic performance of texture analysis and machine learning models for the diagnosis of Covid-19 interstitial pneumonia in Chest X-ray images.

METHODS

Chest X-ray images were accessed from a publicly available repository (https://www.kaggle.com/tawsifurrahman/covid19-radiography-database). Lung areas were manually segmented using a polygonal regions of interest covering both lung areas, using MaZda, a freely available software for texture analysis. A total of 308 features per ROI was extracted. One hundred-ten Covid-19 Chest X-ray images were selected for the final analysis.

RESULTS

Six models, namely NB, GLM, DL, GBT, ANN and PLS-DA were selected and ensembled. According to Youden's index, the Covid-19 Ensemble Machine Learning Score showing the highest Area Under the Curve (0.971±0.015) was 132.57. Assuming this cut-off the Ensemble model performance was estimated evaluating both true and false positive/negative, resulting in 91.8% accuracy with 93% sensitivity and 90% specificity. Moving the cut-off value to -100, although the accuracy resulted lower (90.6%), the Ensemble Machine Learning showed 100% sensitivity, with 80% specificity.

CONCLUSION

Texture analysis of Chest X-ray images and machine learning algorithms may help in differentiating patients with Covid-19 pneumonia. Despite several limitations, this study can lay ground for future researches in this field and help developing more rapid and accurate screening tools for these patients.