Cross-Site Severity Assessment of COVID-19 From CT Images via Domain Adaptation

Cross-site prognosis prediction for nasopharyngeal carcinoma from incomplete multi-modal data

Accurate prognosis prediction for nasopharyngeal carcinoma based on magnetic resonance (MR) images assists in the guidance of treatment intensity, thus reducing the risk of recurrence and death. To reduce repeated labor and sufficiently explore domain knowledge, aggregating labeled/annotated data from external sites enables us to train an intelligent model for a clinical site with unlabeled data. However, this task suffers from the challenges of incomplete multi-modal examination data fusion and image data heterogeneity among sites. This paper proposes a cross-site survival analysis method for prognosis prediction of nasopharyngeal carcinoma from domain adaptation viewpoint. Utilizing a Cox model as the basic framework, our method equips it with a cross-attention based multi-modal fusion regularization. This regularization model effectively fuses the multi-modal information from multi-parametric MR images and clinical features onto a domain-adaptive space, despite the absence of some modalities. To enhance the feature discrimination, we also extend the contrastive learning technique to censored data cases. Compared with the conventional approaches which directly deploy a trained survival model in a new site, our method achieves superior prognosis prediction performance in cross-site validation experiments. These results highlight the key role of cross-site adaptability of our method and support its value in clinical practice.

Deep learning for unsupervised domain adaptation in medical imaging: Recent advancements and future perspectives

Deep learning has demonstrated remarkable performance across various tasks in medical imaging. However, these approaches primarily focus on supervised learning, assuming that the training and testing data are drawn from the same distribution. Unfortunately, this assumption may not always hold true in practice. To address these issues, unsupervised domain adaptation (UDA) techniques have been developed to transfer knowledge from a labeled domain to a related but unlabeled domain. In recent years, significant advancements have been made in UDA, resulting in a wide range of methodologies, including feature alignment, image translation, self-supervision, and disentangled representation methods, among others. In this paper, we provide a comprehensive literature review of recent deep UDA approaches in medical imaging from a technical perspective. Specifically, we categorize current UDA research in medical imaging into six groups and further divide them into finer subcategories based on the different tasks they perform. We also discuss the respective datasets used in the studies to assess the divergence between the different domains. Finally, we discuss emerging areas and provide insights and discussions on future research directions to conclude this survey.

Deep learning for head and neck semi-supervised semantic segmentation

Objective. Radiation therapy (RT) represents a prevalent therapeutic modality for head and neck (H&N) cancer. A crucial phase in RT planning involves the precise delineation of organs-at-risks (OARs), employing computed tomography (CT) scans. Nevertheless, the manual delineation of OARs is a labor-intensive process, necessitating individual scrutiny of each CT image slice, not to mention that a standard CT scan comprises hundreds of such slices. Furthermore, there is a significant domain shift between different institutions' H&N data, which makes traditional semi-supervised learning strategies susceptible to confirmation bias. Therefore, effectively using unlabeled datasets to support annotated datasets for model training has become a critical issue for preventing domain shift and confirmation bias.Approach. In this work, we proposed an innovative cross-domain orthogon-based-perspective consistency (CD-OPC) strategy within a two-branch collaborative training framework, which compels the two sub-networks to acquire valuable features from unrelated perspectives. More specifically, a novel generative pretext task cross-domain prediction (CDP) was designed for learning inherent properties of CT images. Then this prior knowledge was utilized to promote the independent learning of distinct features by the two sub-networks from identical inputs, thereby enhancing the perceptual capabilities of the sub-networks through orthogon-based pseudo-labeling knowledge transfer.Main results. Our CD-OPC model was trained on H&N datasets from nine different institutions, and validated on the four local intuitions' H&N datasets. Among all datasets CD-OPC achieved more advanced performance than other semi-supervised semantic segmentation algorithms.Significance. The CD-OPC method successfully mitigates domain shift and prevents network collapse. In addition, it enhances the network's perceptual abilities, and generates more reliable predictions, thereby further addressing the confirmation bias issue.

Continual Nuclei Segmentation via Prototype-Wise Relation Distillation and Contrastive Learning

Deep learning models have achieved remarkable success in multi-type nuclei segmentation. These models are mostly trained at once with the full annotation of all types of nuclei available, while lack the ability of continually learning new classes due to the problem of catastrophic forgetting. In this paper, we study the practical and important class-incremental continual learning problem, where the model is incrementally updated to new classes without accessing to previous data. We propose a novel continual nuclei segmentation method, to avoid forgetting knowledge of old classes and facilitate the learning of new classes, by achieving feature-level knowledge distillation with prototype-wise relation distillation and contrastive learning. Concretely, prototype-wise relation distillation imposes constraints on the inter-class relation similarity, encouraging the encoder to extract similar class distribution for old classes in the feature space. Prototype-wise contrastive learning with a hard sampling strategy enhances the intra-class compactness and inter-class separability of features, improving the performance on both old and new classes. Experiments on two multi-type nuclei segmentation benchmarks, i.e., MoNuSAC and CoNSeP, demonstrate the effectiveness of our method with superior performance over many competitive methods. Codes are available at https://github.com/zzw-szu/CoNuSeg.

Source Free Semi-Supervised Transfer Learning for Diagnosis of Mental Disorders on fMRI Scans

The high prevalence of mental disorders gradually poses a huge pressure on the public healthcare services. Deep learning-based computer-aided diagnosis (CAD) has emerged to relieve the tension in healthcare institutions by detecting abnormal neuroimaging-derived phenotypes. However, training deep learning models relies on sufficient annotated datasets, which can be costly and laborious. Semi-supervised learning (SSL) and transfer learning (TL) can mitigate this challenge by leveraging unlabeled data within the same institution and advantageous information from source domain, respectively. This work is the first attempt to propose an effective semi-supervised transfer learning (SSTL) framework dubbed S3TL for CAD of mental disorders on fMRI data. Within S3TL, a secure cross-domain feature alignment method is developed to generate target-related source model in SSL. Subsequently, we propose an enhanced dual-stage pseudo-labeling approach to assign pseudo-labels for unlabeled samples in target domain. Finally, an advantageous knowledge transfer method is conducted to improve the generalization capability of the target model. Comprehensive experimental results demonstrate that S3TL achieves competitive accuracies of 69.14%, 69.65%, and 72.62% on ABIDE-I, ABIDE-II, and ADHD-200 datasets, respectively. Furthermore, the simulation experiments also demonstrate the application potential of S3TL through model interpretation analysis and federated learning extension.

The Advent of Domain Adaptation into Artificial Intelligence for Gastrointestinal Endoscopy and Medical Imaging

Artificial intelligence (AI) is a subfield of computer science that aims to implement computer systems that perform tasks that generally require human learning, reasoning, and perceptual abilities. AI is widely used in the medical field. The interpretation of medical images requires considerable effort, time, and skill. AI-aided interpretations, such as automated abnormal lesion detection and image classification, are promising areas of AI. However, when images with different characteristics are extracted, depending on the manufacturer and imaging environment, a so-called domain shift problem occurs in which the developed AI has a poor versatility. Domain adaptation is used to address this problem. Domain adaptation is a tool that generates a newly converted image which is suitable for other domains. It has also shown promise in reducing the differences in appearance among the images collected from different devices. Domain adaptation is expected to improve the reading accuracy of AI for heterogeneous image distributions in gastrointestinal (GI) endoscopy and medical image analyses. In this paper, we review the history and basic characteristics of domain shift and domain adaptation. We also address their use in gastrointestinal endoscopy and the medical field more generally through published examples, perspectives, and future directions.

A Deep Learning Model for the Diagnosis and Discrimination of Gram-Positive and Gram-Negative Bacterial Pneumonia for Children Using Chest Radiography Images and Clinical Information

Purpose

This study aimed to develop a deep learning model based on chest radiography (CXR) images and clinical data to accurately classify gram-positive and gram-negative bacterial pneumonia in children to guide the use of antibiotics.

Methods

We retrospectively collected CXR images along with clinical information for gram-positive (n=447) and gram-negative (n=395) bacterial pneumonia in children from January 1, 2016, to June 30, 2021. Four types of machine learning models based on clinical data and six types of deep learning algorithm models based on image data were constructed, and multi-modal decision fusion was performed.

Results

In the machine learning models, CatBoost, which only used clinical data, had the best performance; its area under the receiver operating characteristic curve (AUC) was significantly higher than that of the other models (P<0.05). The incorporation of clinical information improved the performance of deep learning models that relied solely on image-based classification. Consequently, AUC and F1 increased by 5.6% and 10.2% on average, respectively. The best quality was achieved with ResNet101 (model accuracy: 0.75, recall rate: 0.84, AUC: 0.803, F1: 0.782).

Conclusion

Our study established a pediatric bacterial pneumonia model that utilizes CXR and clinical data to accurately classify cases of gram-negative and gram-positive bacterial pneumonia. The results confirmed that the addition of image data to the convolutional neural network model significantly improved its performance. While the CatBoost-based classifier had greater advantages owing to a smaller dataset, the quality of the Resnet101 model trained using multi-modal data was comparable to that of the CatBoost model, even with a limited number of samples.

Attentive Continuous Generative Self-training for Unsupervised Domain Adaptive Medical Image Translation

Self-training is an important class of unsupervised domain adaptation (UDA) approaches that are used to mitigate the problem of domain shift, when applying knowledge learned from a labeled source domain to unlabeled and heterogeneous target domains. While self-training-based UDA has shown considerable promise on discriminative tasks, including classification and segmentation, through reliable pseudo-label filtering based on the maximum softmax probability, there is a paucity of prior work on self-training-based UDA for generative tasks, including image modality translation. To fill this gap, in this work, we seek to develop a generative self-training (GST) framework for domain adaptive image translation with continuous value prediction and regression objectives. Specifically, we quantify both aleatoric and epistemic uncertainties within our GST using variational Bayes learning to measure the reliability of synthesized data. We also introduce a self-attention scheme that de-emphasizes the background region to prevent it from dominating the training process. The adaptation is then carried out by an alternating optimization scheme with target domain supervision that focuses attention on the regions with reliable pseudo-labels. We evaluated our framework on two cross-scanner/center, inter-subject translation tasks, including tagged-to-cine magnetic resonance (MR) image translation and T1-weighted MR-to-fractional anisotropy translation. Extensive validations with unpaired target domain data showed that our GST yielded superior synthesis performance in comparison to adversarial training UDA methods.

Automatic Classification of Histopathology Images across Multiple Cancers Based on Heterogeneous Transfer Learning

Background: Current artificial intelligence (AI) in histopathology typically specializes on a single task, resulting in a heavy workload of collecting and labeling a sufficient number of images for each type of cancer. Heterogeneous transfer learning (HTL) is expected to alleviate the data bottlenecks and establish models with performance comparable to supervised learning (SL). Methods: An accurate source domain model was trained using 28,634 colorectal patches. Additionally, 1000 sentinel lymph node patches and 1008 breast patches were used to train two target domain models. The feature distribution difference between sentinel lymph node metastasis or breast cancer and CRC was reduced by heterogeneous domain adaptation, and the maximum mean difference between subdomains was used for knowledge transfer to achieve accurate classification across multiple cancers. Result: HTL on 1000 sentinel lymph node patches (L-HTL-1000) outperforms SL on 1000 sentinel lymph node patches (L-SL-1-1000) (average area under the curve (AUC) and standard deviation of L-HTL-1000 vs. L-SL-1-1000: 0.949 ± 0.004 vs. 0.931 ± 0.008, p value = 0.008). There is no significant difference between L-HTL-1000 and SL on 7104 patches (L-SL-2-7104) (0.949 ± 0.004 vs. 0.948 ± 0.008, p value = 0.742). Similar results are observed for breast cancer. B-HTL-1008 vs. B-SL-1-1008: 0.962 ± 0.017 vs. 0.943 ± 0.018, p value = 0.008; B-HTL-1008 vs. B-SL-2-5232: 0.962 ± 0.017 vs. 0.951 ± 0.023, p value = 0.148. Conclusions: HTL is capable of building accurate AI models for similar cancers using a small amount of data based on a large dataset for a certain type of cancer. HTL holds great promise for accelerating the development of AI in histopathology.

COVID-19 imaging, where do we go from here? Bibliometric analysis of medical imaging in COVID-19

OBJECTIVES

We conducted a systematic and comprehensive bibliometric analysis of COVID-19-related medical imaging to determine the current status and indicate possible future directions.

METHODS

This research provides an analysis of Web of Science Core Collection (WoSCC) indexed articles on COVID-19 and medical imaging published between 1 January 2020 and 30 June 2022, using the search terms "COVID-19" and medical imaging terms (such as "X-ray" or "CT"). Publications based solely on COVID-19 themes or medical image themes were excluded. CiteSpace was used to identify the predominant topics and generate a visual map of countries, institutions, authors, and keyword networks.

RESULTS

The search included 4444 publications. The journal with the most publications was European Radiology, and the most co-cited journal was Radiology. China was the most frequently cited country in terms of co-authorship, with the Huazhong University of Science and Technology being the institution contributing with the highest number of relevant co-authorships. Research trends and leading topics included: assessment of initial COVID-19-related clinical imaging features, differential diagnosis using artificial intelligence (AI) technology and model interpretability, diagnosis systems construction, COVID-19 vaccination, complications, and predicting prognosis.

CONCLUSIONS

This bibliometric analysis of COVID-19-related medical imaging helps clarify the current research situation and developmental trends. Subsequent trends in COVID-19 imaging are likely to shift from lung structure to function, from lung tissue to other related organs, and from COVID-19 to the impact of COVID-19 on the diagnosis and treatment of other diseases. Key Points • We conducted a systematic and comprehensive bibliometric analysis of COVID-19-related medical imaging from 1 January 2020 to 30 June 2022. • Research trends and leading topics included assessment of initial COVID-19-related clinical imaging features, differential diagnosis using AI technology and model interpretability, diagnosis systems construction, COVID-19 vaccination, complications, and predicting prognosis. • Future trends in COVID-19-related imaging are likely to involve a shift from lung structure to function, from lung tissue to other related organs, and from COVID-19 to the impact of COVID-19 on the diagnosis and treatment of other diseases.

Quantitative Analysis of Residual COVID-19 Lung CT Features: Consistency among Two Commercial Software

Objective: To investigate two commercial software and their efficacy in the assessment of chest CT sequelae in patients affected by COVID-19 pneumonia, comparing the consistency of tools. Materials and Methods: Included in the study group were 120 COVID-19 patients (56 women and 104 men; 61 years of median age; range: 21–93 years) who underwent chest CT examinations at discharge between 5 March 2020 and 15 March 2021 and again at a follow-up time (3 months; range 30–237 days). A qualitative assessment by expert radiologists in the infectious disease field (experience of at least 5 years) was performed, and a quantitative evaluation using thoracic VCAR software (GE Healthcare, Chicago, Illinois, United States) and a pneumonia module of ANKE ASG-340 CT workstation (HTS Med & Anke, Naples, Italy) was performed. The qualitative evaluation included the presence of ground glass opacities (GGOs) consolidation, interlobular septal thickening, fibrotic-like changes (reticular pattern and/or honeycombing), bronchiectasis, air bronchogram, bronchial wall thickening, pulmonary nodules surrounded by GGOs, pleural and pericardial effusion, lymphadenopathy, and emphysema. A quantitative evaluation included the measurements of GGOs, consolidations, emphysema, residual healthy parenchyma, and total lung volumes for the right and left lung. A chi-square test and non-parametric test were utilized to verify the differences between groups. Correlation coefficients were used to analyze the correlation and variability among quantitative measurements by different computer tools. A receiver operating characteristic (ROC) analysis was performed. Results: The correlation coefficients showed great variability among the quantitative measurements by different tools when calculated on baseline CT scans and considering all patients. Instead, a good correlation (≥0.6) was obtained for the quantitative GGO, as well as the consolidation volumes obtained by two tools when calculated on baseline CT scans, considering the control group. An excellent correlation (≥0.75) was obtained for the quantitative residual healthy lung parenchyma volume, GGO, consolidation volumes obtained by two tools when calculated on follow-up CT scans, and for residual healthy lung parenchyma and GGO quantification when the percentage change of these volumes were calculated between a baseline and follow-up scan. The highest value of accuracy to identify patients with RT-PCR positive compared to the control group was obtained by a GGO total volume quantification by thoracic VCAR (accuracy = 0.75). Conclusions: Computer aided quantification could be an easy and feasible way to assess chest CT sequelae due to COVID-19 pneumonia; however, a great variability among measurements provided by different tools should be considered.

Artificial Intelligence and COVID-19 Using Chest CT Scan and Chest X-ray Images: Machine Learning and Deep Learning Approaches for Diagnosis and Treatment

OBJECTIVE

To report an overview and update on Artificial Intelligence (AI) and COVID-19 using chest Computed Tomography (CT) scan and chest X-ray images (CXR). Machine Learning and Deep Learning Approaches for Diagnosis and Treatment were identified.

METHODS

Several electronic datasets were analyzed. The search covered the years from January 2019 to June 2021. The inclusion criteria were studied evaluating the use of AI methods in COVID-19 disease reporting performance results in terms of accuracy or precision or area under Receiver Operating Characteristic (ROC) curve (AUC).

RESULTS

Twenty-two studies met the inclusion criteria: 13 papers were based on AI in CXR and 10 based on AI in CT. The summarized mean value of the accuracy and precision of CXR in COVID-19 disease were 93.7% ± 10.0% of standard deviation (range 68.4-99.9%) and 95.7% ± 7.1% of standard deviation (range 83.0-100.0%), respectively. The summarized mean value of the accuracy and specificity of CT in COVID-19 disease were 89.1% ± 7.3% of standard deviation (range 78.0-99.9%) and 94.5 ± 6.4% of standard deviation (range 86.0-100.0%), respectively. No statistically significant difference in summarized accuracy mean value between CXR and CT was observed using the Chi square test (p value > 0.05).

CONCLUSIONS

Summarized accuracy of the selected papers is high but there was an important variability; however, less in CT studies compared to CXR studies. Nonetheless, AI approaches could be used in the identification of disease clusters, monitoring of cases, prediction of the future outbreaks, mortality risk, COVID-19 diagnosis, and disease management.