Clinically Applicable AI System for Accurate Diagnosis, Quantitative Measurements, and Prognosis of COVID-19 Pneumonia Using Computed Tomography

Lightweight triangular mesh deformable reconstruction for low quality 3D organ models: Thickness noise and uneven topology

Lightweight triangular mesh models have great potential for real-time 3D visualization of lesions during minimally invasive surgery (MIS). However, the blurred tissue boundaries, high imaging noise, and unoriented points in medical images seriously affect the accuracy and topological quality of surface reconstruction, which can lead to inaccurate lesion localization. In this paper, we present a robust and high-topology-quality triangular mesh reconstruction method that aims to provide a deformable expression model for real-time 3D visualization during surgery. Our approach begins by approximating the model prototype under the guidance of an unsigned distance field by simulating inflation. Then, we introduce a variance-controlled cylindrical domain projection search (VC-CDPS) method to achieve the final surface fitting. Additionally, we incorporate topology optimization into the iterative reconstruction process to ensure smoothness and good topology of the reconstruction model. To validate our method, we conduct experiments on a geometric model with high noise and a human organ model manually segmented by novice doctors. The results demonstrate that our reconstructed model exhibits better surface quality and noise immunity. Furthermore, we conduct a comparison experiment of model deformation and propose a metric to measure the topological quality of the model. Through in vitro tissue experiments, we explored the relationship between topological quality and deformation accuracy. The results reveal a positive correlation between deformation accuracy and topological quality.

Deep learning-based system for automatic identification of benign and malignant eyelid tumours

AIMS

Our aim is to develop a deep learning-based system for automatically identifying and classifying benign and malignant tumours of the eyelid to improve diagnostic accuracy and efficiency.

METHODS

The dataset includes photographs of normal eyelids, benign and malignant eyelid tumours and was randomly divided into a training and validation dataset in a ratio of 8:2. We used the training dataset to train eight convolutional neural network models to classify normal eyelids, benign and malignant eyelid tumours. These models included VGG16, ResNet50, Inception-v4, EfficientNet-V2-M and their variants. The validation dataset was used to evaluate and compare the performance of the different deep learning models.

RESULTS

All eight models achieved an average accuracy greater than 0.746 for identifying normal eyelids, benign and malignant eyelid tumours, with an average sensitivity and specificity exceeding 0.790 and 0.866, respectively. The mean area under the receiver operating characteristic curve (AUC) for the eight models was more than 0.904 in correctly identifying normal eyelids, benign and malignant eyelid tumours. The dual-path Inception-v4 network demonstrated the highest performance, with an AUC of 0.930 (95% CI 0.900 to 0.954) and an F1-score of 0.838 (95% CI 0.787 to 0.882).

CONCLUSION

The deep learning-based system shows significant potential in improving the diagnosis of eyelid tumours, providing a reliable and efficient tool for clinical practice. Future work will validate the model with more extensive and diverse datasets and integrate it into clinical workflows for real-time diagnostic support.

MetaGP: A generative foundation model integrating electronic health records and multimodal imaging for addressing unmet clinical needs

Artificial intelligence makes strides in specialized diagnostics but faces challenges in complex clinical scenarios, such as rare disease diagnosis and emergency condition identification. To address these limitations, we develop Meta General Practitioner (MetaGP), a 32-billion-parameter generative foundation model trained on extensive datasets, including over 8 million electronic health records, biomedical literature, and medical textbooks. MetaGP demonstrates robust diagnostic capabilities, achieving accuracy comparable to experienced clinicians. In rare disease cases, it achieves an average diagnostic score of 1.57, surpassing GPT-4's 0.93. For emergency conditions, it improves diagnostic accuracy for junior and mid-level clinicians by 53% and 46%, respectively. MetaGP also excels in generating medical imaging reports, producing high-quality outputs for chest X-rays and computed tomography, often rated comparable to or superior to physician-authored reports. These findings highlight MetaGP's potential to transform clinical decision-making across diverse medical contexts.

Generation of short-term follow-up chest CT images using a latent diffusion model in COVID-19

PURPOSE

Despite a global decrease in the number of COVID-19 patients, early prediction of the clinical course for optimal patient care remains challenging. Recently, the usefulness of image generation for medical images has been investigated. This study aimed to generate short-term follow-up chest CT images using a latent diffusion model in patients with COVID-19.

MATERIALS AND METHODS

We retrospectively enrolled 505 patients with COVID-19 for whom the clinical parameters (patient background, clinical symptoms, and blood test results) upon admission were available and chest CT imaging was performed. Subject datasets (n = 505) were allocated for training (n = 403), and the remaining (n = 102) were reserved for evaluation. The image underwent variational autoencoder (VAE) encoding, resulting in latent vectors. The information consisting of initial clinical parameters and radiomic features were formatted as a table data encoder. Initial and follow-up latent vectors and the initial table data encoders were utilized for training the diffusion model. The evaluation data were used to generate prognostic images. Then, similarity of the prognostic images (generated images) and the follow-up images (real images) was evaluated by zero-mean normalized cross-correlation (ZNCC), peak signal-to-noise ratio (PSNR), and structural similarity (SSIM). Visual assessment was also performed using a numerical rating scale.

RESULTS

Prognostic chest CT images were generated using the diffusion model. Image similarity showed reasonable values of 0.973 ± 0.028 for the ZNCC, 24.48 ± 3.46 for the PSNR, and 0.844 ± 0.075 for the SSIM. Visual evaluation of the images by two pulmonologists and one radiologist yielded a reasonable mean score.

CONCLUSIONS

The similarity and validity of generated predictive images for the course of COVID-19-associated pneumonia using a diffusion model were reasonable. The generation of prognostic images may suggest potential utility for early prediction of the clinical course in COVID-19-associated pneumonia and other respiratory diseases.

Unified total body CT image with multiple organ specific windowings: validating improved diagnostic accuracy and speed in trauma cases

Total-body CT scans are useful in saving trauma patients; however, interpreting numerous images with varied window settings slows injury detection. We developed an algorithm for "unified total-body CT image with multiple organ-specific windowings (Uni-CT)", and assessing its impact on physician accuracy and speed in trauma CT interpretation. From November 7, 2008, to June 19, 2020, 40 cases of total-body CT images for blunt trauma with multiple injuries, were collected from the emergency department of Osaka General Medical Center and randomly divided into two groups. In half of the cases, the Uni-CT algorithm using semantic segmentation assigned visibility-friendly window settings to each organ. Four physicians with varying levels of experience interpreted 20 cases using the algorithm and 20 cases in conventional settings. The performance was analyzed based on the accuracy, sensitivity, specificity of the target findings, and diagnosis speed. In the proposal and conventional groups, patients had an average of 2.6 and 2.5 targeting findings, mean ages of 51.8 and 57.7 years, and male proportions of 60% and 45%, respectively. The agreement rate for physicians' diagnoses was κ = 0.70. Average accuracy, sensitivity, and specificity of target findings were 84.8%, 74.3%, 96.9% and 85.5%, 81.2%, 91.5%, respectively, with no significant differences. Diagnostic speed per case averaged 71.9 and 110.4 s in each group (p < 0.05). The Uni-CT algorithm improved the diagnostic speed of total-body CT for trauma, maintaining accuracy comparable to that of conventional methods.

Leveraging deep transfer learning and explainable AI for accurate COVID-19 diagnosis: Insights from a multi-national chest CT scan study

The COVID-19 pandemic has emerged as a global health crisis, impacting millions worldwide. Although chest computed tomography (CT) scan images are pivotal in diagnosing COVID-19, their manual interpretation by radiologists is time-consuming and potentially subjective. Automated computer-aided diagnostic (CAD) frameworks offer efficient and objective solutions. However, machine or deep learning methods often face challenges in their reproducibility due to underlying biases and methodological flaws. To address these issues, we propose XCT-COVID, an explainable, transferable, and reproducible CAD framework based on deep transfer learning to predict COVID-19 infection from CT scan images accurately. This is the first study to develop three distinct models within a unified framework by leveraging a previously unexplored large dataset and two widely used smaller datasets. We employed five known convolutional neural network architectures, both with and without pretrained weights, on the larger dataset. We optimized hyperparameters through extensive grid search and 5-fold cross-validation (CV), significantly enhancing the model performance. Experimental results from the larger dataset showed that the VGG16 architecture (XCT-COVID-L) with pretrained weights consistently outperformed other architectures, achieving the best performance, on both 5-fold CV and independent test. When evaluated with the external datasets, XCT-COVID-L performed well with data with similar distributions, demonstrating its transferability. However, its performance significantly decreased on smaller datasets with lower-quality images. To address this, we developed other models, XCT-COVID-S1 and XCT-COVID-S2, specifically for the smaller datasets, outperforming existing methods. Moreover, eXplainable Artificial Intelligence (XAI) analyses were employed to interpret the models' functionalities. For prediction and reproducibility purposes, the implementation of XCT-COVID is publicly accessible at https://github.com/cbbl-skku-org/XCT-COVID/.

Assessing the impact of comorbidities on disease severity in COVID-19 patients requires consideration of age

Older age and comorbidities are risk factors for increased coronavirus disease 2019 (COVID-19) severity, but few studies have explored their interaction. This study aimed to assess the actual impacts of these factors on disease severity in COVID-19. The enrolled COVID-19 patients were divided into 4 age subgroups (≤44, 45-59, 60-74, and ≥75 years). Logistic regression analysis was conducted to determine the association between comorbidities and disease severity; Kappa consistency test was implemented to verify the study results. Of the 1663 patients with COVID-19, 287 had severe disease. The disease severity was correlated with the age-adjusted Charlson Comorbidity Index in each age group. In the 4 subgroups, the odds ratio of age-adjusted Charlson Comorbidity Index declined with age. After removing age interference, diabetes and cardio-cerebrovascular diseases were the main risk factors for severe disease in patients aged <75 years, whereas only chronic lung disease was associated with disease severity in patients aged ≥75 years. When comorbidities alone were used to predict disease severity, only the predictions were consistent with real outcomes in patients aged ≥75 years, compared with the predictions of high-risk comorbidities mentioned in World Health Organization and Chinese guidelines (Kappa 0.106, P < .05). Although older age and comorbidities were risk factors for severe COVID-19, their effects on disease severity varied across age groups. Additionally, comorbidities had a greater impact on COVID-19 severity in younger patients.

Foundation models for fast, label-free detection of glioma infiltration

A critical challenge in glioma treatment is detecting tumour infiltration during surgery to achieve safe maximal resection1-3. Unfortunately, safely resectable residual tumour is found in the majority of patients with glioma after surgery, causing early recurrence and decreased survival4-6. Here we present FastGlioma, a visual foundation model for fast (<10 s) and accurate detection of glioma infiltration in fresh, unprocessed surgical tissue. FastGlioma was pretrained using large-scale self-supervision (around 4 million images) on rapid, label-free optical microscopy, and fine-tuned to output a normalized score that indicates the degree of tumour infiltration within whole-slide optical images. In a prospective, multicentre, international testing cohort of patients with diffuse glioma (n = 220), FastGlioma was able to detect and quantify the degree of tumour infiltration with an average area under the receiver operating characteristic curve of 92.1 ± 0.9%. FastGlioma outperformed image-guided and fluorescence-guided adjuncts for detecting tumour infiltration during surgery by a wide margin in a head-to-head, prospective study (n = 129). The performance of FastGlioma remained high across diverse patient demographics, medical centres and diffuse glioma molecular subtypes as defined by the World Health Organization. FastGlioma shows zero-shot generalization to other adult and paediatric brain tumour diagnoses, demonstrating the potential for our foundation model to be used as a general-purpose adjunct for guiding brain tumour surgeries. These findings represent the transformative potential of medical foundation models to unlock the role of artificial intelligence in the care of patients with cancer.

Large Language Models for Mental Health Applications: Systematic Review

BACKGROUND

Large language models (LLMs) are advanced artificial neural networks trained on extensive datasets to accurately understand and generate natural language. While they have received much attention and demonstrated potential in digital health, their application in mental health, particularly in clinical settings, has generated considerable debate.

OBJECTIVE

This systematic review aims to critically assess the use of LLMs in mental health, specifically focusing on their applicability and efficacy in early screening, digital interventions, and clinical settings. By systematically collating and assessing the evidence from current studies, our work analyzes models, methodologies, data sources, and outcomes, thereby highlighting the potential of LLMs in mental health, the challenges they present, and the prospects for their clinical use.

METHODS

Adhering to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, this review searched 5 open-access databases: MEDLINE (accessed by PubMed), IEEE Xplore, Scopus, JMIR, and ACM Digital Library. Keywords used were (mental health OR mental illness OR mental disorder OR psychiatry) AND (large language models). This study included articles published between January 1, 2017, and April 30, 2024, and excluded articles published in languages other than English.

RESULTS

In total, 40 articles were evaluated, including 15 (38%) articles on mental health conditions and suicidal ideation detection through text analysis, 7 (18%) on the use of LLMs as mental health conversational agents, and 18 (45%) on other applications and evaluations of LLMs in mental health. LLMs show good effectiveness in detecting mental health issues and providing accessible, destigmatized eHealth services. However, assessments also indicate that the current risks associated with clinical use might surpass their benefits. These risks include inconsistencies in generated text; the production of hallucinations; and the absence of a comprehensive, benchmarked ethical framework.

CONCLUSIONS

This systematic review examines the clinical applications of LLMs in mental health, highlighting their potential and inherent risks. The study identifies several issues: the lack of multilingual datasets annotated by experts, concerns regarding the accuracy and reliability of generated content, challenges in interpretability due to the "black box" nature of LLMs, and ongoing ethical dilemmas. These ethical concerns include the absence of a clear, benchmarked ethical framework; data privacy issues; and the potential for overreliance on LLMs by both physicians and patients, which could compromise traditional medical practices. As a result, LLMs should not be considered substitutes for professional mental health services. However, the rapid development of LLMs underscores their potential as valuable clinical aids, emphasizing the need for continued research and development in this area.

TRIAL REGISTRATION

PROSPERO CRD42024508617; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=508617.

Identifying Potential Factors Associated With Racial Disparities in COVID-19 Outcomes: Retrospective Cohort Study Using Machine Learning on Real-World Data

BACKGROUND

Racial disparities in COVID-19 incidence and outcomes have been widely reported. Non-Hispanic Black patients endured worse outcomes disproportionately compared with non-Hispanic White patients, but the epidemiological basis for these observations was complex and multifaceted.

OBJECTIVE

This study aimed to elucidate the potential reasons behind the worse outcomes of COVID-19 experienced by non-Hispanic Black patients compared with non-Hispanic White patients and how these variables interact using an explainable machine learning approach.

METHODS

In this retrospective cohort study, we examined 28,943 laboratory-confirmed COVID-19 cases from the OneFlorida Research Consortium's data trust of health care recipients in Florida through April 28, 2021. We assessed the prevalence of pre-existing comorbid conditions, geo-socioeconomic factors, and health outcomes in the structured electronic health records of COVID-19 cases. The primary outcome was a composite of hospitalization, intensive care unit admission, and mortality at index admission. We developed and validated a machine learning model using Extreme Gradient Boosting to evaluate predictors of worse outcomes of COVID-19 and rank them by importance.

RESULTS

Compared to non-Hispanic White patients, non-Hispanic Blacks patients were younger, more likely to be uninsured, had a higher prevalence of emergency department and inpatient visits, and were in regions with higher area deprivation index rankings and pollutant concentrations. Non-Hispanic Black patients had the highest burden of comorbidities and rates of the primary outcome. Age was a key predictor in all models, ranking highest in non-Hispanic White patients. However, for non-Hispanic Black patients, congestive heart failure was a primary predictor. Other variables, such as food environment measures and air pollution indicators, also ranked high. By consolidating comorbidities into the Elixhauser Comorbidity Index, this became the top predictor, providing a comprehensive risk measure.

CONCLUSIONS

The study reveals that individual and geo-socioeconomic factors significantly influence the outcomes of COVID-19. It also highlights varying risk profiles among different racial groups. While these findings suggest potential disparities, further causal inference and statistical testing are needed to fully substantiate these observations. Recognizing these relationships is vital for creating effective, tailored interventions that reduce disparities and enhance health outcomes across all racial and socioeconomic groups.

Phenotypic, Metabolic and Genetic Adaptations of the Ficus Species to Abiotic Stress Response: A Comprehensive Review

The Ficus genus, having radiated from the tropics and subtropics to the temperate zone worldwide, is the largest genus among woody plants, comprising over 800 species. Evolution of the Ficus species results in genetic diversity, global radiation and geographical differentiations, suggesting adaption to diverse environments and coping with stresses. Apart from familiar physiological changes, such as stomatal closure and alteration in plant hormone levels, the Ficus species exhibit a unique mechanism in response to abiotic stress, such as regulation of leaf temperature and retention of drought memory. The stress-resistance genes harbored by Ficus result in effective responses to abiotic stress. Understanding the stress-resistance mechanisms in Ficus provides insights into the genetic breeding toward stress-tolerant crop cultivars. Following upon these issues, we comprehensively reviewed recent progress concerning the Ficus genes and relevant mechanisms that play important roles in the abiotic stress responses. These highlight prospectively important application potentials of the stress-resistance genes in Ficus.

Determination of prognostic markers for COVID-19 disease severity using routine blood tests and machine learning

The need for the identification of risk factors associated to COVID-19 disease severity remains urgent. Patients' care and resource allocation can be potentially different and are defined based on the current classification of disease severity. This classification is based on the analysis of clinical parameters and routine blood tests, which are not standardized across the globe. Some laboratory test alterations have been associated to COVID-19 severity, although these data are conflicting partly due to the different methodologies used across different studies. This study aimed to construct and validate a disease severity prediction model using machine learning (ML). Seventy-two patients admitted to a Brazilian hospital and diagnosed with COVID-19 through RT-PCR and/or ELISA, and with varying degrees of disease severity, were included in the study. Their electronic medical records and the results from daily blood tests were used to develop a ML model to predict disease severity. Using the above data set, a combination of five laboratorial biomarkers was identified as accurate predictors of COVID-19 severe disease with a ROC-AUC of 0.80 ​±​ 0.13. Those biomarkers included prothrombin activity, ferritin, serum iron, ATTP and monocytes. The application of the devised ML model may help rationalize clinical decision and care.

"KAIZEN" method realizing implementation of deep-learning models for COVID-19 CT diagnosis in real world hospitals

Numerous COVID-19 diagnostic imaging Artificial Intelligence (AI) studies exist. However, none of their models were of potential clinical use, primarily owing to methodological defects and the lack of implementation considerations for inference. In this study, all development processes of the deep-learning models are performed based on strict criteria of the "KAIZEN checklist", which is proposed based on previous AI development guidelines to overcome the deficiencies mentioned above. We develop and evaluate two binary-classification deep-learning models to triage COVID-19: a slice model examining a Computed Tomography (CT) slice to find COVID-19 lesions; a series model examining a series of CT images to find an infected patient. We collected 2,400,200 CT slices from twelve emergency centers in Japan. Area Under Curve (AUC) and accuracy were calculated for classification performance. The inference time of the system that includes these two models were measured. For validation data, the slice and series models recognized COVID-19 with AUCs and accuracies of 0.989 and 0.982, 95.9% and 93.0% respectively. For test data, the models' AUCs and accuracies were 0.958 and 0.953, 90.0% and 91.4% respectively. The average inference time per case was 2.83 s. Our deep-learning system realizes accuracy and inference speed high enough for practical use. The systems have already been implemented in four hospitals and eight are under progression. We released an application software and implementation code for free in a highly usable state to allow its use in Japan and globally.

Impact of Artificial Intelligence (AI) Technology in Healthcare Sector: A Critical Evaluation of Both Sides of the Coin

The influence of artificial intelligence (AI) has drastically risen in recent years, especially in the field of medicine. Its influence has spread so greatly that it is determined to become a pillar in the future medical world. A comprehensive literature search related to AI in healthcare was performed in the PubMed database and retrieved the relevant information from suitable ones. AI excels in aspects such as rapid adaptation, high diagnostic accuracy, and data management that can help improve workforce productivity. With this potential in sight, the FDA has continuously approved more machine learning (ML) software to be used by medical workers and scientists. However, there are few controversies such as increased chances of data breaches, concern for clinical implementation, and potential healthcare dilemmas. In this article, the positive and negative aspects of AI implementation in healthcare are discussed, as well as recommended some potential solutions to the potential issues at hand.

Artificial Intelligence in battling infectious diseases: A transformative role

It is widely acknowledged that infectious diseases have wrought immense havoc on human society, being regarded as adversaries from which humanity cannot elude. In recent years, the advancement of Artificial Intelligence (AI) technology has ushered in a revolutionary era in the realm of infectious disease prevention and control. This evolution encompasses early warning of outbreaks, contact tracing, infection diagnosis, drug discovery, and the facilitation of drug design, alongside other facets of epidemic management. This article presents an overview of the utilization of AI systems in the field of infectious diseases, with a specific focus on their role during the COVID-19 pandemic. The article also highlights the contemporary challenges that AI confronts within this domain and posits strategies for their mitigation. There exists an imperative to further harness the potential applications of AI across multiple domains to augment its capacity in effectively addressing future disease outbreaks.

Prediction of oxygen supplementation by a deep-learning model integrating clinical parameters and chest CT images in COVID-19

PURPOSE

As of March 2023, the number of patients with COVID-19 worldwide is declining, but the early diagnosis of patients requiring inpatient treatment and the appropriate allocation of limited healthcare resources remain unresolved issues. In this study we constructed a deep-learning (DL) model to predict the need for oxygen supplementation using clinical information and chest CT images of patients with COVID-19.

MATERIALS AND METHODS

We retrospectively enrolled 738 patients with COVID-19 for whom clinical information (patient background, clinical symptoms, and blood test findings) was available and chest CT imaging was performed. The initial data set was divided into 591 training and 147 evaluation data. We developed a DL model that predicted oxygen supplementation by integrating clinical information and CT images. The model was validated at two other facilities (n = 191 and n = 230). In addition, the importance of clinical information for prediction was assessed.

RESULTS

The proposed DL model showed an area under the curve (AUC) of 89.9% for predicting oxygen supplementation. Validation from the two other facilities showed an AUC > 80%. With respect to interpretation of the model, the contribution of dyspnea and the lactate dehydrogenase level was higher in the model.

CONCLUSIONS

The DL model integrating clinical information and chest CT images had high predictive accuracy. DL-based prediction of disease severity might be helpful in the clinical management of patients with COVID-19.

Deep learning on longitudinal CT scans: automated prediction of treatment outcomes in hospitalized tuberculosis patients

Three deep learning (DL)-based prediction models (PMs) using longitudinal CT images were developed to predict tuberculosis (TB) treatment outcomes. The internal dataset consists of 493 bacteriologically confirmed TB patients who completed the anti-tuberculosis treatment with three-time CT scans, including a pretreatment CT scan and two follow-up CT scans. PM1 was trained using only pretreatment CT scans, and PM2 and PM3 were developed by adding follow-up scans. An independent testing was performed on external dataset comprising 86 TB patients. The area under the curve for classifying success and drug-resistant (DR)-TB was improved on both internal (0.609 vs. 0.625 vs. 0.815) and external (0.627 vs. 0.705 vs. 0.735) dataset by adding follow-up scans. The accuracy and F1-score also showed an increasing tendency in the external test. Regular follow-up CT scans can aid in the treatment prediction, and special attention should be given to early intensive phase of treatment to identify high-risk DR-TB patients.

Applying the digital data and the bioinformatics tools in SARS-CoV-2 research

Bioinformatics has been playing a crucial role in the scientific progress to fight against the pandemic of the coronavirus disease 2019 (COVID-19) caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). The advances in novel algorithms, mega data technology, artificial intelligence and deep learning assisted the development of novel bioinformatics tools to analyze daily increasing SARS-CoV-2 data in the past years. These tools were applied in genomic analyses, evolutionary tracking, epidemiological analyses, protein structure interpretation, studies in virus-host interaction and clinical performance. To promote the in-silico analysis in the future, we conducted a review which summarized the databases, web services and software applied in SARS-CoV-2 research. Those digital resources applied in SARS-CoV-2 research may also potentially contribute to the research in other coronavirus and non-coronavirus viruses.

Artificial intelligence in ophthalmology: The path to the real-world clinic

Artificial intelligence (AI) has great potential to transform healthcare by enhancing the workflow and productivity of clinicians, enabling existing staff to serve more patients, improving patient outcomes, and reducing health disparities. In the field of ophthalmology, AI systems have shown performance comparable with or even better than experienced ophthalmologists in tasks such as diabetic retinopathy detection and grading. However, despite these quite good results, very few AI systems have been deployed in real-world clinical settings, challenging the true value of these systems. This review provides an overview of the current main AI applications in ophthalmology, describes the challenges that need to be overcome prior to clinical implementation of the AI systems, and discusses the strategies that may pave the way to the clinical translation of these systems.

Enhancing thoracic disease detection using chest X-rays from PubMed Central Open Access

Large chest X-rays (CXR) datasets have been collected to train deep learning models to detect thorax pathology on CXR. However, most CXR datasets are from single-center studies and the collected pathologies are often imbalanced. The aim of this study was to automatically construct a public, weakly-labeled CXR database from articles in PubMed Central Open Access (PMC-OA) and to assess model performance on CXR pathology classification by using this database as additional training data. Our framework includes text extraction, CXR pathology verification, subfigure separation, and image modality classification. We have extensively validated the utility of the automatically generated image database on thoracic disease detection tasks, including Hernia, Lung Lesion, Pneumonia, and pneumothorax. We pick these diseases due to their historically poor performance in existing datasets: the NIH-CXR dataset (112,120 CXR) and the MIMIC-CXR dataset (243,324 CXR). We find that classifiers fine-tuned with additional PMC-CXR extracted by the proposed framework consistently and significantly achieved better performance than those without (e.g., Hernia: 0.9335 vs 0.9154; Lung Lesion: 0.7394 vs. 0.7207; Pneumonia: 0.7074 vs. 0.6709; Pneumothorax 0.8185 vs. 0.7517, all in AUC with p< 0.0001) for CXR pathology detection. In contrast to previous approaches that manually submit the medical images to the repository, our framework can automatically collect figures and their accompanied figure legends. Compared to previous studies, the proposed framework improved subfigure segmentation and incorporates our advanced self-developed NLP technique for CXR pathology verification. We hope it complements existing resources and improves our ability to make biomedical image data findable, accessible, interoperable, and reusable.

Predicting gene mutation status via artificial intelligence technologies based on multimodal integration (MMI) to advance precision oncology

Personalized treatment strategies for cancer frequently rely on the detection of genetic alterations which are determined by molecular biology assays. Historically, these processes typically required single-gene sequencing, next-generation sequencing, or visual inspection of histopathology slides by experienced pathologists in a clinical context. In the past decade, advances in artificial intelligence (AI) technologies have demonstrated remarkable potential in assisting physicians with accurate diagnosis of oncology image-recognition tasks. Meanwhile, AI techniques make it possible to integrate multimodal data such as radiology, histology, and genomics, providing critical guidance for the stratification of patients in the context of precision therapy. Given that the mutation detection is unaffordable and time-consuming for a considerable number of patients, predicting gene mutations based on routine clinical radiological scans or whole-slide images of tissue with AI-based methods has become a hot issue in actual clinical practice. In this review, we synthesized the general framework of multimodal integration (MMI) for molecular intelligent diagnostics beyond standard techniques. Then we summarized the emerging applications of AI in the prediction of mutational and molecular profiles of common cancers (lung, brain, breast, and other tumor types) pertaining to radiology and histology imaging. Furthermore, we concluded that there truly exist multiple challenges of AI techniques in the way of its real-world application in the medical field, including data curation, feature fusion, model interpretability, and practice regulations. Despite these challenges, we still prospect the clinical implementation of AI as a highly potential decision-support tool to aid oncologists in future cancer treatment management.

Prototype early diagnostic model for invasive pulmonary aspergillosis based on deep learning and big data training

BACKGROUND

Currently, the diagnosis of invasive pulmonary aspergillosis (IPA) mainly depends on the integration of clinical, radiological and microbiological data. Artificial intelligence (AI) has shown great advantages in dealing with data-rich biological and medical challenges, but the literature on IPA diagnosis is rare.

OBJECTIVE

This study aimed to provide a non-invasive, objective and easy-to-use AI approach for the early diagnosis of IPA.

METHODS

We generated a prototype diagnostic deep learning model (IPA-NET) comprising three interrelated computation modules for the automatic diagnosis of IPA. First, IPA-NET was subjected to transfer learning using 300,000 CT images of non-fungal pneumonia from an online database. Second, training and internal test sets, including clinical features and chest CT images of patients with IPA and non-fungal pneumonia in the early stage of the disease, were independently constructed for model training and internal verification. Third, the model was further validated using an external test set.

RESULTS

IPA-NET showed a marked diagnostic performance for IPA as verified by the internal test set, with an accuracy of 96.8%, a sensitivity of 0.98, a specificity of 0.96 and an area under the curve (AUC) of 0.99. When further validated using the external test set, IPA-NET showed an accuracy of 89.7%, a sensitivity of 0.88, a specificity of 0.91 and an AUC of 0.95.

CONCLUSION

This novel deep learning model provides a non-invasive, objective and reliable method for the early diagnosis of IPA.

COVID-19 lung infection segmentation from chest CT images based on CAPA-ResUNet

Coronavirus disease 2019 (COVID-19) epidemic has devastating effects on personal health around the world. It is significant to achieve accurate segmentation of pulmonary infection regions, which is an early indicator of disease. To solve this problem, a deep learning model, namely, the content-aware pre-activated residual UNet (CAPA-ResUNet), was proposed for segmenting COVID-19 lesions from CT slices. In this network, the pre-activated residual block was used for down-sampling to solve the problems of complex foreground and large fluctuations of distribution in datasets during training and to avoid gradient disappearance. The area loss function based on the false segmentation regions was proposed to solve the problem of fuzzy boundary of the lesion area. This model was evaluated by the public dataset (COVID-19 Lung CT Lesion Segmentation Challenge-2020) and compared its performance with those of classical models. Our method gains an advantage over other models in multiple metrics. Such as the Dice coefficient, specificity (Spe), and intersection over union (IoU), our CAPA-ResUNet obtained 0.775 points, 0.972 points, and 0.646 points, respectively. The Dice coefficient of our model was 2.51% higher than Content-aware residual UNet (CARes-UNet). The code is available at https://github.com/malu108/LungInfectionSeg.

Hybrid intelligent model for classifying chest X-ray images of COVID-19 patients using genetic algorithm and neutrosophic logic

The highly spreading virus, COVID-19, created a huge need for an accurate and speedy diagnosis method. The famous RT-PCR test is costly and not available for many suspected cases. This article proposes a neurotrophic model to diagnose COVID-19 patients based on their chest X-ray images. The proposed model has five main phases. First, the speeded up robust features (SURF) method is applied to each X-ray image to extract robust invariant features. Second, three sampling algorithms are applied to treat imbalanced dataset. Third, the neutrosophic rule-based classification system is proposed to generate a set of rules based on the three neutrosophic values < T; I; F>, the degrees of truth, indeterminacy falsity. Fourth, a genetic algorithm is applied to select the optimal neutrosophic rules to improve the classification performance. Fifth, in this phase, the classification-based neutrosophic logic is proposed. The testing rule matrix is constructed with no class label, and the goal of this phase is to determine the class label for each testing rule using intersection percentage between testing and training rules. The proposed model is referred to as GNRCS. It is compared with six state-of-the-art classifiers such as multilayer perceptron (MLP), support vector machines (SVM), linear discriminant analysis (LDA), decision tree (DT), naive Bayes (NB), and random forest classifiers (RFC) with quality measures of accuracy, precision, sensitivity, specificity, and F1-score. The results show that the proposed model is powerful for COVID-19 recognition with high specificity and high sensitivity and less computational complexity. Therefore, the proposed GNRCS model could be used for real-time automatic early recognition of COVID-19.

Habitat Imaging Biomarkers for Diagnosis and Prognosis in Cancer Patients Infected with COVID-19

OBJECTIVES

Cancer patients have worse outcomes from the COVID-19 infection and greater need for ventilator support and elevated mortality rates than the general population. However, previous artificial intelligence (AI) studies focused on patients without cancer to develop diagnosis and severity prediction models. Little is known about how the AI models perform in cancer patients. In this study, we aim to develop a computational framework for COVID-19 diagnosis and severity prediction particularly in a cancer population and further compare it head-to-head to a general population.

METHODS

We have enrolled multi-center international cohorts with 531 CT scans from 502 general patients and 420 CT scans from 414 cancer patients. In particular, the habitat imaging pipeline was developed to quantify the complex infection patterns by partitioning the whole lung regions into phenotypically different subregions. Subsequently, various machine learning models nested with feature selection were built for COVID-19 detection and severity prediction.

RESULTS

These models showed almost perfect performance in COVID-19 infection diagnosis and predicting its severity during cross validation. Our analysis revealed that models built separately on the cancer population performed significantly better than those built on the general population and locked to test on the cancer population. This may be because of the significant difference among the habitat features across the two different cohorts.

CONCLUSIONS

Taken together, our habitat imaging analysis as a proof-of-concept study has highlighted the unique radiologic features of cancer patients and demonstrated effectiveness of CT-based machine learning model in informing COVID-19 management in the cancer population.

CCT: Lightweight compact convolutional transformer for lung disease CT image classification

Computed tomography (CT) imaging results are an important criterion for the diagnosis of lung disease. CT images can clearly show the characteristics of lung lesions. Early and accurate detection of lung diseases helps clinicians to improve patient care effectively. Therefore, in this study, we used a lightweight compact convolutional transformer (CCT) to build a prediction model for lung disease classification using chest CT images. We added a position offset term and changed the attention mechanism of the transformer encoder to an axial attention mechanism module. As a result, the classification performance of the model was improved in terms of height and width. We show that the model effectively classifies COVID-19, community pneumonia, and normal conditions on the CC-CCII dataset. The proposed model outperforms other comparable models in the test set, achieving an accuracy of 98.5% and a sensitivity of 98.6%. The results show that our method achieves a larger field of perception on CT images, which positively affects the classification of CT images. Thus, the method can provide adequate assistance to clinicians.

A novel deep learning-based method for COVID-19 pneumonia detection from CT images

Background

The sensitivity of RT-PCR in diagnosing COVID-19 is only 60–70%, and chest CT plays an indispensable role in the auxiliary diagnosis of COVID-19 pneumonia, but the results of CT imaging are highly dependent on professional radiologists.

Aims

This study aimed to develop a deep learning model to assist radiologists in detecting COVID-19 pneumonia.

Methods

The total study population was 437. The training dataset contained 26,477, 2468, and 8104 CT images of normal, CAP, and COVID-19, respectively. The validation dataset contained 14,076, 1028, and 3376 CT images of normal, CAP, and COVID-19 patients, respectively. The test set included 51 normal cases, 28 CAP patients, and 51 COVID-19 patients. We designed and trained a deep learning model to recognize normal, CAP, and COVID-19 patients based on U-Net and ResNet-50. Moreover, the diagnoses of the deep learning model were compared with different levels of radiologists.

Results

In the test set, the sensitivity of the deep learning model in diagnosing normal cases, CAP, and COVID-19 patients was 98.03%, 89.28%, and 92.15%, respectively. The diagnostic accuracy of the deep learning model was 93.84%. In the validation set, the accuracy was 92.86%, which was better than that of two novice doctors (86.73% and 87.75%) and almost equal to that of two experts (94.90% and 93.88%). The AI model performed significantly better than all four radiologists in terms of time consumption (35 min vs. 75 min, 93 min, 79 min, and 82 min).

Conclusion

The AI model we obtained had strong decision-making ability, which could potentially assist doctors in detecting COVID-19 pneumonia.

Dynamic feature learning for COVID-19 segmentation and classification

Since December 2019, coronavirus SARS-CoV-2 (COVID-19) has rapidly developed into a global epidemic, with millions of patients affected worldwide. As part of the diagnostic pathway, computed tomography (CT) scans are used to help patient management. However, parenchymal imaging findings in COVID-19 are non-specific and can be seen in other diseases. In this work, we propose to first segment lesions from CT images, and further, classify COVID-19 patients from healthy persons and common pneumonia patients. In detail, a novel Dynamic Fusion Segmentation Network (DFSN) that automatically segments infection-related pixels is first proposed. Within this network, low-level features are aggregated to high-level ones to effectively capture context characteristics of infection regions, and high-level features are dynamically fused to model multi-scale semantic information of lesions. Based on DFSN, Dynamic Transfer-learning Classification Network (DTCN) is proposed to distinguish COVID-19 patients. Within DTCN, a pre-trained DFSN is transferred and used as the backbone to extract pixel-level information. Then the pixel-level information is dynamically selected and used to make a diagnosis. In this way, the pre-trained DFSN is utilized through transfer learning, and clinical significance of segmentation results is comprehensively considered. Thus DTCN becomes more sensitive to typical signs of COVID-19. Extensive experiments are conducted to demonstrate effectiveness of the proposed DFSN and DTCN frameworks. The corresponding results indicate that these two models achieve state-of-the-art performance in terms of segmentation and classification.

Jaya-tunicate swarm algorithm based generative adversarial network for COVID-19 prediction with chest computed tomography images

A novel corona virus (COVID-19) has materialized as the respiratory syndrome in recent decades. Chest computed tomography scanning is the significant technology for monitoring and predicting COVID-19. To predict the patients of COVID-19 at early stage poses an open challenge in the research community. Therefore, an effective prediction mechanism named Jaya-tunicate swarm algorithm driven generative adversarial network (Jaya-TSA with GAN) is proposed in this research to find patients of COVID-19 infections. The developed Jaya-TSA is the incorporation of Jaya algorithm with tunicate swarm algorithm (TSA). However, lungs lobs are segmented using Bayesian fuzzy clustering, which effectively find the boundary regions of lung lobes. Based on the extracted features, the process of COVID-19 prediction is accomplished using GAN. The optimal solution is obtained by training GAN using proposed Jaya-TSA with respect to fitness measure. The dimensionality of features is reduced by extracting the optimal features, which enable to increase the speed of training process. Moreover, the developed Jaya-TSA based GAN attained outstanding effectiveness by considering the factors, like, specificity, accuracy, and sensitivity that captured the importance as 0.8857, 0.8727, and 0.85 by varying training data.

Radiogenomic System for Non-Invasive Identification of Multiple Actionable Mutations and PD-L1 Expression in Non-Small Cell Lung Cancer Based on CT Images

PURPOSE

Personalized treatments such as targeted therapy and immunotherapy have revolutionized the predominantly therapeutic paradigm for non-small cell lung cancer (NSCLC). However, these treatment decisions require the determination of targetable genomic and molecular alterations through invasive genetic or immunohistochemistry (IHC) tests. Numerous previous studies have demonstrated that artificial intelligence can accurately predict the single-gene status of tumors based on radiologic imaging, but few studies have achieved the simultaneous evaluation of multiple genes to reflect more realistic clinical scenarios.

METHODS

We proposed a multi-label multi-task deep learning (MMDL) system for non-invasively predicting actionable NSCLC mutations and PD-L1 expression utilizing routinely acquired computed tomography (CT) images. This radiogenomic system integrated transformer-based deep learning features and radiomic features of CT volumes from 1096 NSCLC patients based on next-generation sequencing (NGS) and IHC tests.

RESULTS

For each task cohort, we randomly split the corresponding dataset into training (80%), validation (10%), and testing (10%) subsets. The area under the receiver operating characteristic curves (AUCs) of the MMDL system achieved 0.862 (95% confidence interval (CI), 0.758-0.969) for discrimination of a panel of 8 mutated genes, including EGFR, ALK, ERBB2, BRAF, MET, ROS1, RET and KRAS, 0.856 (95% CI, 0.663-0.948) for identification of a 10-molecular status panel (previous 8 genes plus TP53 and PD-L1); and 0.868 (95% CI, 0.641-0.972) for classifying EGFR / PD-L1 subtype, respectively.

CONCLUSIONS

To the best of our knowledge, this study is the first deep learning system to simultaneously analyze 10 molecular expressions, which might be utilized as an assistive tool in conjunction with or in lieu of ancillary testing to support precision treatment options.

AI for COVID-19 Detection from Radiographs: Incisive Analysis of State of the Art Techniques, Key Challenges and Future Directions

Background and objective

In recent years, Artificial Intelligence has had an evident impact on the way research addresses challenges in different domains. It has proven to be a huge asset, especially in the medical field, allowing for time-efficient and reliable solutions. This research aims to spotlight the impact of deep learning and machine learning models in the detection of COVID-19 from medical images. This is achieved by conducting a review of the state-of-the-art approaches proposed by the recent works in this field.

Methods

The main focus of this study is the recent developments of classification and segmentation approaches to image-based COVID-19 detection. The study reviews 140 research papers published in different academic research databases. These papers have been screened and filtered based on specified criteria, to acquire insights prudent to image-based COVID-19 detection.

Results

The methods discussed in this review include different types of imaging modality, predominantly X-rays and CT scans. These modalities are used for classification and segmentation tasks as well. This review seeks to categorize and discuss the different deep learning and machine learning architectures employed for these tasks, based on the imaging modality utilized. It also hints at other possible deep learning and machine learning architectures that can be proposed for better results towards COVID-19 detection. Along with that, a detailed overview of the emerging trends and breakthroughs in Artificial Intelligence-based COVID-19 detection has been discussed as well.

Conclusion

This work concludes by stipulating the technical and non-technical challenges faced by researchers and illustrates the advantages of image-based COVID-19 detection with Artificial Intelligence techniques.

SARS-CoV-2 and Endothelial Cells: Vascular Changes, Intussusceptive Microvascular Growth and Novel Therapeutic Windows

Endothelial activation in infectious diseases plays a crucial role in understanding and predicting the outcomes and future treatments of several clinical conditions. COVID-19 is no exception. Moving from basic principles to novel approaches, an evolving view of endothelial activation provides insights into a better knowledge of the upstream actors in COVID-19 as a crucial future direction for managing SARS-CoV-2 and other infections. Assessing the function of resting and damaged endothelial cells in infection, particularly in COVID-19, five critical processes emerged controlling thrombo-resistance: vascular integrity, blood flow regulation, immune cell trafficking, angiogenesis and intussusceptive microvascular growth. Endothelial cell injury is associated with thrombosis, increased vessel contraction and a crucial phenomenon identified as intussusceptive microvascular growth, an unprecedented event of vessel splitting into two lumens through the integration of circulating pro-angiogenic cells. An essential awareness of endothelial cells and their phenotypic changes in COVID-19 inflammation is pivotal to understanding the vascular biology of infections and may offer crucial new therapeutic windows.

Enabling image optimisation and artificial intelligence technologies for better Internet of Things framework to predict COVID

Sensor technology advancements have provided a viable solution to fight COVID and to develop healthcare systems based on Internet of Things (IoTs). In this study, image processing and Artificial Intelligence (AI) are used to improve the IoT framework. Computed Tomography (CT) image‐based forecasting of COVID disease is among the important activities in medicine for measuring the severity of variability in the human body. In COVID CT images, the optimal gamma correction value was optimised using the Whale Optimisation Algorithm (WOA). During the search for the optimal solution, WOA was found to be a highly efficient algorithm, which has the characteristics of high precision and fast convergence. Whale Optimisation Algorithm is used to find best gamma correction value to present detailed information about a lung CT image, Also, in this study, analysis of important AI techniques has been done, such as Support Vector Machine (SVM) and Deep‐Learning (Deep‐Learning (DL)) for COVID disease forecasting in terms of amount of data training and computational power. Many experiments have been implemented to investigate the optimisation: SVM and DL with WOA and without WOA are compared by using confusion matrix parameters. From the results, we find that the DL model outperforms the SVM with WOA and without WOA.

Development and validation of an abnormality-derived deep-learning diagnostic system for major respiratory diseases

Respiratory diseases impose a tremendous global health burden on large patient populations. In this study, we aimed to develop DeepMRD^TR, a deep learning-based medical image interpretation system for the diagnosis of major respiratory diseases based on the automated identification of a wide range of radiological abnormalities through computed tomography (CT) and chest X-ray (CXR) from real-world, large-scale datasets. DeepMRD^TR comprises four networks (two CT-Nets and two CXR-Nets) that exploit contrastive learning to generate pre-training parameters that are fine-tuned on the retrospective dataset collected from a single institution. The performance of DeepMRD^TR was evaluated for abnormality identification and disease diagnosis on data from two different institutions: one was an internal testing dataset from the same institution as the training data and the second was collected from an external institution to evaluate the model generalizability and robustness to an unrelated population dataset. In such a difficult multi-class diagnosis task, our system achieved the average area under the receiver operating characteristic curve (AUC) of 0.856 (95% confidence interval (CI):0.843–0.868) and 0.841 (95%CI:0.832–0.887) for abnormality identification, and 0.900 (95%CI:0.872–0.958) and 0.866 (95%CI:0.832–0.887) for major respiratory diseases’ diagnosis on CT and CXR datasets, respectively. Furthermore, to achieve a clinically actionable diagnosis, we deployed a preliminary version of DeepMRD^TR into the clinical workflow, which was performed on par with senior experts in disease diagnosis, with an AUC of 0.890 and a Cohen’s k of 0.746–0.877 at a reasonable timescale; these findings demonstrate the potential to accelerate the medical workflow to facilitate early diagnosis as a triage tool for respiratory diseases which supports improved clinical diagnoses and decision-making.

The Application of Artificial Intelligence in the Diagnosis and Drug Resistance Prediction of Pulmonary Tuberculosis

With the increasing incidence and mortality of pulmonary tuberculosis, in addition to tough and controversial disease management, time-wasting and resource-limited conventional approaches to the diagnosis and differential diagnosis of tuberculosis are still awkward issues, especially in countries with high tuberculosis burden and backwardness. In the meantime, the climbing proportion of drug-resistant tuberculosis poses a significant hazard to public health. Thus, auxiliary diagnostic tools with higher efficiency and accuracy are urgently required. Artificial intelligence (AI), which is not new but has recently grown in popularity, provides researchers with opportunities and technical underpinnings to develop novel, precise, rapid, and automated implements for pulmonary tuberculosis care, including but not limited to tuberculosis detection. In this review, we aimed to introduce representative AI methods, focusing on deep learning and radiomics, followed by definite descriptions of the state-of-the-art AI models developed using medical images and genetic data to detect pulmonary tuberculosis, distinguish the infection from other pulmonary diseases, and identify drug resistance of tuberculosis, with the purpose of assisting physicians in deciding the appropriate therapeutic schedule in the early stage of the disease. We also enumerated the challenges in maximizing the impact of AI in this field such as generalization and clinical utility of the deep learning models.

Deep learning predicts immune checkpoint inhibitor-related pneumonitis from pretreatment computed tomography images

Immune checkpoint inhibitors (ICIs) have revolutionized the treatment of lung cancer, including both non-small cell lung cancer and small cell lung cancer. Despite the promising results of immunotherapies, ICI-related pneumonitis (ICIP) is a potentially fatal adverse event. Therefore, early detection of patients at risk for developing ICIP before the initiation of immunotherapy is critical for alleviating future complications with early interventions and improving treatment outcomes. In this study, we present the first reported work that explores the potential of deep learning to predict patients who are at risk for developing ICIP. To this end, we collected the pretreatment baseline CT images and clinical information of 24 patients who developed ICIP after immunotherapy and 24 control patients who did not. A multimodal deep learning model was constructed based on 3D CT images and clinical data. To enhance performance, we employed two-stage transfer learning by pre-training the model sequentially on a large natural image dataset and a large CT image dataset, as well as transfer learning. Extensive experiments were conducted to verify the effectiveness of the key components used in our method. Using five-fold cross-validation, our method accurately distinguished ICIP patients from non-ICIP patients, with area under the receiver operating characteristic curve of 0.918 and accuracy of 0.920. This study demonstrates the promising potential of deep learning to identify patients at risk for developing ICIP. The proposed deep learning model enables efficient risk stratification, close monitoring, and prompt management of ICIP, ultimately leading to better treatment outcomes.

The value of longitudinal clinical data and paired CT scans in predicting the deterioration of COVID-19 revealed by an artificial intelligence system

The respective value of clinical data and CT examinations in predicting COVID-19 progression is unclear, because the CT scans and clinical data previously used are not synchronized in time. To address this issue, we collected 119 COVID-19 patients with 341 longitudinal CT scans and paired clinical data, and we developed an AI system for the prediction of COVID-19 deterioration. By combining features extracted from CT and clinical data with our system, we can predict whether a patient will develop severe symptoms during hospitalization. Complementary to clinical data, CT examinations show significant add-on values for the prediction of COVID-19 progression in the early stage of COVID-19, especially in the 6^th to 8^th day after the symptom onset, indicating that this is the ideal time window for the introduction of CT examinations. We release our AI system to provide clinicians with additional assistance to optimize CT usage in the clinical workflow.

•

COVID-19 patients with 341 longitudinal CT scans and paired clinical data included

•

A new AI model for the prediction of COVID-19 progression was developed

•

CT scans show significant add-on value over clinical data for the prediction

•

Day 6–8 after the onset of COVID-19 symptoms is an ideal time window for a CT scan

Challenges of deep learning methods for COVID-19 detection using public datasets

Since the COVID-19 pandemic, several research studies have proposed Deep Learning (DL)-based automated COVID-19 detection, reporting high cross-validation accuracy when classifying COVID-19 patients from normal or other common Pneumonia. Although the reported outcomes are very high in most cases, these results were obtained without an independent test set from a separate data source(s). DL models are likely to overfit training data distribution when independent test sets are not utilized or are prone to learn dataset-specific artifacts rather than the actual disease characteristics and underlying pathology. This study aims to assess the promise of such DL methods and datasets by investigating the key challenges and issues by examining the compositions of the available public image datasets and designing different experimental setups. A convolutional neural network-based network, called CVR-Net (COVID-19 Recognition Network), has been proposed for conducting comprehensive experiments to validate our hypothesis. The presented end-to-end CVR-Net is a multi-scale-multi-encoder ensemble model that aggregates the outputs from two different encoders and their different scales to convey the final prediction probability. Three different classification tasks, such as 2-, 3-, 4-classes, are designed where the train-test datasets are from the single, multiple, and independent sources. The obtained binary classification accuracy is 99.8% for a single train-test data source, where the accuracies fall to 98.4% and 88.7% when multiple and independent train-test data sources are utilized. Similar outcomes are noticed in multi-class categorization tasks for single, multiple, and independent data sources, highlighting the challenges in developing DL models with the existing public datasets without an independent test set from a separate dataset. Such a result concludes a requirement for a better-designed dataset for developing DL tools applicable in actual clinical settings. The dataset should have an independent test set; for a single machine or hospital source, have a more balanced set of images for all the prediction classes; and have a balanced dataset from several hospitals and demography. Our source codes and model are publicly available for the research community for further improvements.

An original deep learning model using limited data for COVID-19 discrimination: A multi-center study

OBJECTIVES

Artificial intelligence (AI) has been proved to be a highly efficient tool for COVID-19 diagnosis, but the large data size and heavy label force required for algorithm development and the poor generalizability of AI algorithms, to some extent, limit the application of AI technology in clinical practice. The aim of this study is to develop an AI algorithm with high robustness using limited chest CT data for COVID-19 discrimination.

METHODS

A three dimensional algorithm that combined multi-instance learning (MIL) with the long and short-term memory (LSTM) architecture (3DMTM) was developed for differentiating COVID-19 from community acquired pneumonia (CAP) while logistic regression (LR), k-nearest neighbor (KNN), support vector machine (SVM) and a three dimensional convolutional neural network (3D CNN) set for comparison. Totally, 515 patients with or without COVID-19 between December 2019 and March 2020 from 5 different hospitals were recruited and divided into relatively large (150 COVID-19 and 183 CAP cases) and relatively small datasets (17 COVID-19 and 35 CAP cases) for either training or validation and another independent dataset (37 COVID-19 and 93 CAP cases) for external test. Area under the receiver operating characteristic curve (AUC), sensitivity, specificity, precision, accuracy, F1 score, and G-Mean were utilized for performance evaluation.

RESULTS

In the external test cohort, the relatively large data-based 3DMTM-LD achieved an AUC of 0.956 (95%CI, 0.929∼0.982) with 86.2% and 98.0% for its sensitivity and specificity. 3DMTM-SD got an AUC of 0.937 (95%CI, 0.909∼0.965) while the AUC of 3DCM-SD decreased dramatically to 0.714 (95%CI, 0.649∼0.780) with training data reduction. KNN-MMSD, LR-MMSD, SVM-MMSD and 3DCM-MMSD benefited significantly from the inclusion of clinical information while models trained with relatively large dataset got slight performance improvement in COVID-19 discrimination. 3DMTM, trained with either CT or multi-modal data, presented comparably excellent performance in COVID-19 discrimination.

CONCLUSIONS

The 3DMTM algorithm presented excellent robustness for COVID-19 discrimination with limited CT data. 3DMTM based on CT data performed comparably in COVID-19 discrimination with that trained with multi-modal information. Clinical information could improve the performance of KNN, LR, SVM and 3DCM in COVID-19 discrimination, especially in the scenario with limited data for training. This article is protected by copyright. All rights reserved.

Artificial intelligence to detect malignant eyelid tumors from photographic images

Malignant eyelid tumors can invade adjacent structures and pose a threat to vision and even life. Early identification of malignant eyelid tumors is crucial to avoiding substantial morbidity and mortality. However, differentiating malignant eyelid tumors from benign ones can be challenging for primary care physicians and even some ophthalmologists. Here, based on 1,417 photographic images from 851 patients across three hospitals, we developed an artificial intelligence system using a faster region-based convolutional neural network and deep learning classification networks to automatically locate eyelid tumors and then distinguish between malignant and benign eyelid tumors. The system performed well in both internal and external test sets (AUCs ranged from 0.899 to 0.955). The performance of the system is comparable to that of a senior ophthalmologist, indicating that this system has the potential to be used at the screening stage for promoting the early detection and treatment of malignant eyelid tumors.

Robust weakly supervised learning for COVID-19 recognition using multi-center CT images

The world is currently experiencing an ongoing pandemic of an infectious disease named coronavirus disease 2019 (i.e., COVID-19), which is caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Computed Tomography (CT) plays an important role in assessing the severity of the infection and can also be used to identify those symptomatic and asymptomatic COVID-19 carriers. With a surge of the cumulative number of COVID-19 patients, radiologists are increasingly stressed to examine the CT scans manually. Therefore, an automated 3D CT scan recognition tool is highly in demand since the manual analysis is time-consuming for radiologists and their fatigue can cause possible misjudgment. However, due to various technical specifications of CT scanners located in different hospitals, the appearance of CT images can be significantly different leading to the failure of many automated image recognition approaches. The multi-domain shift problem for the multi-center and multi-scanner studies is therefore nontrivial that is also crucial for a dependable recognition and critical for reproducible and objective diagnosis and prognosis. In this paper, we proposed a COVID-19 CT scan recognition model namely coronavirus information fusion and diagnosis network (CIFD-Net) that can efficiently handle the multi-domain shift problem via a new robust weakly supervised learning paradigm. Our model can resolve the problem of different appearance in CT scan images reliably and efficiently while attaining higher accuracy compared to other state-of-the-art methods.

Metabolomic analyses reveal new stage-specific features of COVID-19

The current pandemic of coronavirus disease 2019 (COVID-19) has affected >160 million individuals to date, and has caused millions of deaths worldwide, at least in part due to the unclarified pathophysiology of this disease. Identifying the underlying molecular mechanisms of COVID-19 is critical to overcome this pandemic. Metabolites mirror the disease progression of an individual and can provide extensive insights into their pathophysiological significance at each stage of disease. We provide a comprehensive view of metabolic characterisation of sera from COVID-19 patients at all stages using untargeted and targeted metabolomic analysis. As compared with the healthy controls, we observed different alteration patterns of circulating metabolites from the mild, severe and recovery stages, in both the discovery cohort and the validation cohort, which suggests that metabolic reprogramming of glucose metabolism and the urea cycle are potential pathological mechanisms for COVID-19 progression. Our findings suggest that targeting glucose metabolism and the urea cycle may be a viable approach to fight COVID-19 at various stages along the disease course.

Artificial intelligence for COVID-19: battling the pandemic with computational intelligence

The new coronavirus disease 2019 (COVID-19) has become a global pandemic leading to over 180 million confirmed cases and nearly 4 million deaths until June 2021, according to the World Health Organization. Since the initial report in December 2019 , COVID-19 has demonstrated a high transmission rate (with an R0 > 2), a diverse set of clinical characteristics (e.g., high rate of hospital and intensive care unit admission rates, multi-organ dysfunction for critically ill patients due to hyperinflammation, thrombosis, etc.), and a tremendous burden on health care systems around the world. To understand the serious and complex diseases and develop effective control, treatment, and prevention strategies, researchers from different disciplines have been making significant efforts from different aspects including epidemiology and public health, biology and genomic medicine, as well as clinical care and patient management. In recent years, artificial intelligence (AI) has been introduced into the healthcare field to aid clinical decision-making for disease diagnosis and treatment such as detecting cancer based on medical images, and has achieved superior performance in multiple data-rich application scenarios. In the COVID-19 pandemic, AI techniques have also been used as a powerful tool to overcome the complex diseases. In this context, the goal of this study is to review existing studies on applications of AI techniques in combating the COVID-19 pandemic. Specifically, these efforts can be grouped into the fields of epidemiology, therapeutics, clinical research, social and behavioral studies and are summarized. Potential challenges, directions, and open questions are discussed accordingly, which may provide new insights into addressing the COVID-19 pandemic and would be helpful for researchers to explore more related topics in the post-pandemic era.

One Shot Model For The Prediction of COVID-19 And Lesions Segmentation In Chest CT Scans Through The Affinity Among Lesion Mask Features

We present a novel framework that integrates segmentation of lesion masks and prediction of COVID-19 in chest CT scans in one shot. In order to classify the whole input image, we introduce a type of associations among lesion mask features extracted from the scan slice that we refer to as affinities. First, we map mask features to the affinity space by training an affinity matrix. Next, we map them back into the feature space through a trainable affinity vector. Finally, this feature representation is used for the classification of the whole input scan slice. We achieve a 93.55% COVID-19 sensitivity, 96.93% common pneumonia sensitivity, 99.37% true negative rate and 97.37% F1-score on the test split of CNCB-NCOV dataset with 21192 chest CT scan slices. We also achieve a 0.4240 mean average precision on the lesion segmentation task. All source code, models and results are publicly available on https://github.com/AlexTS1980/COVID-Affinity-Model.

COVID-CT-Mask-Net: prediction of COVID-19 from CT scans using regional features

We present COVID-CT-Mask-Net model that predicts COVID-19 in chest CT scans. The model works in two stages: in the first stage, Mask R-CNN is trained to localize and detect two types of lesions in images. In the second stage, these detections are fused to classify the whole input image. To develop the solution for the three-class problem (COVID-19, Common Pneumonia and Control), we used the COVIDx-CT data split derived from the dataset of chest CT scans collected by China National Center for Bioinformation. We use 3000 images (about 5% of the train split of COVIDx-CT) to train the model. Without any complicated data normalization, balancing and regularization, and training only a small fraction of the model's parameters, we achieve a 9 0 . 8 0 % COVID-19 sensitivity, 9 1 . 6 2 % Common Pneumonia sensitivity and 9 2 . 1 0 % true negative rate (Control sensitivity), an overall accuracy of 9 1 . 6 6 % and F1-score of 9 1 . 5 0 % on the test data split with 21192 images, bringing the ratio of test to train data to 7.06. We also establish an important result that regional predictions (bounding boxes with confidence scores) detected by Mask R-CNN can be used to classify whole images. The full source code, models and pretrained weights are available on https://github.com/AlexTS1980/COVID-CT-Mask-Net.

Nanobioengineering: A promising approach for early detection of COVID-19

Unique pneumonia due to an unknown source emerged in December 2019 in the city of Wuhan, China. Consequently, the World Health Organization (WHO) declared this condition as a new coronavirus disease-19 also known as COVID-19 on February 11, 2020, which on March 13, 2020 was declared as a pandemic. The virus that causes COVID-19 was found to have a similar genome (80% similarity) with the previously known acute respiratory syndrome also known as SARS-CoV. The novel virus was later named Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). SARS-CoV-2 falls in the family of Coronaviridae which is further divided into Nidovirales and another subfamily called Orthocoronavirinae. The four generations of the coronaviruses belongs to the Orthocoronavirinae family that consists of alpha, beta, gamma and delta coronavirus which are denoted as α-CoV, β-CoV, γ-CoV, δ-CoV respectively. The α-CoV and β-CoVs are mainly known to infect mammals whereas γ-CoV and δ-CoV are generally found in birds. The β-CoVs also comprise of SARS-CoV and also include another virus that was found in the Middle East called the Middle East respiratory syndrome virus (MERS-CoV) and the cause of current pandemic SARS-CoV-2. These viruses initially cause the development of pneumonia in the patients and further development of a severe case of acute respiratory distress syndrome (ARDS) and other related symptoms that can be fatal leading to death.

COLI-Net: Deep learning-assisted fully automated COVID-19 lung and infection pneumonia lesion detection and segmentation from chest computed tomography images

We present a deep learning (DL)-based automated whole lung and COVID-19 pneumonia infectious lesions (COLI-Net) detection and segmentation from chest computed tomography (CT) images. This multicenter/multiscanner study involved 2368 (347'259 2D slices) and 190 (17 341 2D slices) volumetric CT exams along with their corresponding manual segmentation of lungs and lesions, respectively. All images were cropped, resized, and the intensity values clipped and normalized. A residual network with non-square Dice loss function built upon TensorFlow was employed. The accuracy of lung and COVID-19 lesions segmentation was evaluated on an external reverse transcription-polymerase chain reaction positive COVID-19 dataset (7'333 2D slices) collected at five different centers. To evaluate the segmentation performance, we calculated different quantitative metrics, including radiomic features. The mean Dice coefficients were 0.98 ± 0.011 (95% CI, 0.98-0.99) and 0.91 ± 0.038 (95% CI, 0.90-0.91) for lung and lesions segmentation, respectively. The mean relative Hounsfield unit differences were 0.03 ± 0.84% (95% CI, -0.12 to 0.18) and -0.18 ± 3.4% (95% CI, -0.8 to 0.44) for the lung and lesions, respectively. The relative volume difference for lung and lesions were 0.38 ± 1.2% (95% CI, 0.16-0.59) and 0.81 ± 6.6% (95% CI, -0.39 to 2), respectively. Most radiomic features had a mean relative error less than 5% with the highest mean relative error achieved for the lung for the range first-order feature (-6.95%) and least axis length shape feature (8.68%) for lesions. We developed an automated DL-guided three-dimensional whole lung and infected regions segmentation in COVID-19 patients to provide fast, consistent, robust, and human error immune framework for lung and pneumonia lesion detection and quantification.

Unbox the black-box for the medical explainable AI via multi-modal and multi-centre data fusion: A mini-review, two showcases and beyond

Explainable Artificial Intelligence (XAI) is an emerging research topic of machine learning aimed at unboxing how AI systems' black-box choices are made. This research field inspects the measures and models involved in decision-making and seeks solutions to explain them explicitly. Many of the machine learning algorithms cannot manifest how and why a decision has been cast. This is particularly true of the most popular deep neural network approaches currently in use. Consequently, our confidence in AI systems can be hindered by the lack of explainability in these black-box models. The XAI becomes more and more crucial for deep learning powered applications, especially for medical and healthcare studies, although in general these deep neural networks can return an arresting dividend in performance. The insufficient explainability and transparency in most existing AI systems can be one of the major reasons that successful implementation and integration of AI tools into routine clinical practice are uncommon. In this study, we first surveyed the current progress of XAI and in particular its advances in healthcare applications. We then introduced our solutions for XAI leveraging multi-modal and multi-centre data fusion, and subsequently validated in two showcases following real clinical scenarios. Comprehensive quantitative and qualitative analyses can prove the efficacy of our proposed XAI solutions, from which we can envisage successful applications in a broader range of clinical questions.

CovH2SD: A COVID-19 detection approach based on Harris Hawks Optimization and stacked deep learning

Starting from Wuhan in China at the end of 2019, coronavirus disease (COVID-19) has propagated fast all over the world, affecting the lives of billions of people and increasing the mortality rate worldwide in few months. The golden treatment against the invasive spread of COVID-19 is done by identifying and isolating the infected patients, and as a result, fast diagnosis of COVID-19 is a critical issue. The common laboratory test for confirming the infection of COVID-19 is Reverse Transcription Polymerase Chain Reaction (RT-PCR). However, these tests suffer from some problems in time, accuracy, and availability. Chest images have proven to be a powerful tool in the early detection of COVID-19. In the current study, a hybrid learning and optimization approach named CovH2SD is proposed for the COVID-19 detection from the Chest Computed Tomography (CT) images. CovH2SD uses deep learning and pre-trained models to extract the features from the CT images and learn from them. It uses Harris Hawks Optimization (HHO) algorithm to optimize the hyperparameters. Transfer learning is applied using nine pre-trained convolutional neural networks (i.e. ResNet50, ResNet101, VGG16, VGG19, Xception, MobileNetV1, MobileNetV2, DenseNet121, and DenseNet169). Fast Classification Stage (FCS) and Compact Stacking Stage (CSS) are suggested to stack the best models into a single one. Nine experiments are applied and results are reported based on the Loss, Accuracy, Precision, Recall, F1-Score, and Area Under Curve (AUC) performance metrics. The comparison between combinations is applied using the Weighted Sum Method (WSM). Six experiments report a WSM value above 96.5%. The top WSM and accuracy reported values are 99.31% and 99.33% respectively which are higher than the eleven compared state-of-the-art studies.