Domain generalization on medical imaging classification using episodic training with task augmentation

Advancing precise diagnosis of nasopharyngeal carcinoma through endoscopy-based radiomics analysis

Nasopharyngeal carcinoma (NPC) has high metastatic potential and is hard to detect early. This study aims to develop a deep learning model for NPC diagnosis using optical imagery. From April 2008 to May 2021, we analyzed 12,087 nasopharyngeal endoscopic images and 309 videos from 1,108 patients. The pretrained model was fine-tuned with stochastic gradient descent on the final layers. Data augmentation was applied during training. Videos were converted to images for malignancy scoring. Performance metrics like AUC, accuracy, and sensitivity were calculated based on the malignancy score. The deep learning model demonstrated high performance in identifying NPC, with AUC values of 0.981 (95% confidence of interval [CI] 0.965-0.996) for the Fujian Cancer Hospital dataset and 0.937 (0.905-0.970) for the Jiangxi Cancer Hospital dataset. The proposed model effectively diagnoses NPC with high accuracy, sensitivity, and specificity across multiple datasets. It shows promise for early NPC detection, especially in identifying latent lesions.

SAMCF: Adaptive global style alignment and multi-color spaces fusion for joint optic cup and disc segmentation

The optic cup (OC) and optic disc (OD) are two critical structures in retinal fundus images, and their relative positions and sizes are essential for effectively diagnosing eye diseases. With the success of deep learning in computer vision, deep learning-based segmentation models have been widely used for joint optic cup and disc segmentation. However, there are three prominent issues that impact the segmentation performance. First, significant differences among datasets collecting from various institutions, protocols, and devices lead to performance degradation of models. Second, we find that images with only RGB information struggle to counteract the interference caused by brightness variations, affecting color representation capability. Finally, existing methods typically ignored the edge perception, facing the challenges in obtaining clear and smooth edge segmentation results. To address these drawbacks, we propose a novel framework based on Style Alignment and Multi-Color Fusion (SAMCF) for joint OC and OD segmentation. Initially, we introduce a domain generalization method to generate uniformly styled images without damaged image content for mitigating domain shift issues. Next, based on multiple color spaces, we propose a feature extraction and fusion network aiming to handle brightness variation interference and improve color representation capability. Lastly, an edge aware loss is designed to generate fine edge segmentation results. Our experiments conducted on three public datasets, DGS, RIM, and REFUGE, demonstrate that our proposed SAMCF achieves superior performance to existing state-of-the-art methods. Moreover, SAMCF exhibits remarkable generalization ability across multiple retinal fundus image datasets, showcasing its outstanding generality.

Patient Re-Identification Based on Deep Metric Learning in Trunk Computed Tomography Images Acquired from Devices from Different Vendors

During radiologic interpretation, radiologists read patient identifiers from the metadata of medical images to recognize the patient being examined. However, it is challenging for radiologists to identify "incorrect" metadata and patient identification errors. We propose a method that uses a patient re-identification technique to link correct metadata to an image set of computed tomography images of a trunk with lost or wrongly assigned metadata. This method is based on a feature vector matching technique that uses a deep feature extractor to adapt to the cross-vendor domain contained in the scout computed tomography image dataset. To identify "incorrect" metadata, we calculated the highest similarity score between a follow-up image and a stored baseline image linked to the correct metadata. The re-identification performance tests whether the image with the highest similarity score belongs to the same patient, i.e., whether the metadata attached to the image are correct. The similarity scores between the follow-up and baseline images for the same "correct" patients were generally greater than those for "incorrect" patients. The proposed feature extractor was sufficiently robust to extract individual distinguishable features without additional training, even for unknown scout computed tomography images. Furthermore, the proposed augmentation technique further improved the re-identification performance of the subset for different vendors by incorporating changes in width magnification due to changes in patient table height during each examination. We believe that metadata checking using the proposed method would help detect the metadata with an "incorrect" patient identifier assigned due to unavoidable errors such as human error.

Immunotherapy efficacy prediction through a feature re-calibrated 2.5D neural network

BACKGROUND AND OBJECTIVE

Lung cancer continues to be a leading cause of cancer-related mortality worldwide, with immunotherapy emerging as a promising therapeutic strategy for advanced non-small cell lung cancer (NSCLC). Despite its potential, not all patients experience benefits from immunotherapy, and the current biomarkers used for treatment selection possess inherent limitations. As a result, the implementation of imaging-based biomarkers to predict the efficacy of lung cancer treatments offers a promising avenue for improving therapeutic outcomes.

METHODS

This study presents an automatic system for immunotherapy efficacy prediction on the subjects with lung cancer, facilitating significant clinical implications. Our model employs an advanced 2.5D neural network that incorporates 2D intra-slice feature extraction and 3D inter-slice feature aggregation. We further present a lesion-focused prior to guide the re-calibration for intra-slice features, and a attention-based re-calibration for the inter-slice features. Finally, we design an accumulated back-propagation strategy to optimize network parameters in a memory-efficient fashion.

RESULTS

We demonstrate that the proposed method achieves impressive performance on an in-house clinical dataset, surpassing existing state-of-the-art models. Furthermore, the proposed model exhibits increased efficiency in inference for each subject on average. To further validate the effectiveness of our model and its components, we conducted comprehensive and in-depth ablation experiments and discussions.

CONCLUSION

The proposed model showcases the potential to enhance physicians' diagnostic performance due to its impressive performance in predicting immunotherapy efficacy, thereby offering significant clinical application value. Moreover, we conduct adequate comparison experiments of the proposed methods and existing advanced models. These findings contribute to our understanding of the proposed model's effectiveness and serve as motivation for future work in immunotherapy efficacy prediction.

A causality-inspired generalized model for automated pancreatic cancer diagnosis

Pancreatic cancer (PC) is a severely malignant cancer variant with high mortality. Since PC has no obvious symptoms, most PC patients are belatedly diagnosed at advanced disease stages. Recently, artificial intelligence (AI) approaches have demonstrated promising prospects for early diagnosis of pancreatic cancer. However, certain non-causal factors (such as intensity and texture appearance variations, also called confounders) tend to induce spurious correlation with PC diagnosis. This undermines the generalization performance and the clinical applicability of the AI-based PC diagnosis approaches. Therefore, we propose a causal intervention based automated method for pancreatic cancer diagnosis with contrast-enhanced computerized tomography (CT) images, where a confounding effects reduction scheme is developed for alleviating spurious correlations to achieve unbiased learning, thereby improving the generalization performance. Specifically, a continuous image generation strategy was developed to simulate wide variations of intensity differences caused by imaging heterogeneities, where Monte Carlo sampling is added to further enhance the continuity of simulated images. Then, to enhance the pancreatic texture variability, a texture diversification method was introduced in conjunction with gradient-based data augmentation. Finally, a causal intervention strategy was proposed to alleviate the adverse confounding effects by decoupling the causal and non-causal factors and combining them randomly. Extensive experiments showed remarkable diagnosis performance on a cross-validation dataset. Also, promising generalization performance with an average accuracy of 0.87 was attained on three independent test sets of a total of 782 subjects. Therefore, the proposed method shows high clinical feasibility and applicability for pancreatic cancer diagnosis.

CDDSA: Contrastive domain disentanglement and style augmentation for generalizable medical image segmentation

Generalization to previously unseen images with potential domain shifts is essential for clinically applicable medical image segmentation. Disentangling domain-specific and domain-invariant features is key for Domain Generalization (DG). However, existing DG methods struggle to achieve effective disentanglement. To address this problem, we propose an efficient framework called Contrastive Domain Disentanglement and Style Augmentation (CDDSA) for generalizable medical image segmentation. First, a disentangle network decomposes the image into domain-invariant anatomical representation and domain-specific style code, where the former is sent for further segmentation that is not affected by domain shift, and the disentanglement is regularized by a decoder that combines the anatomical representation and style code to reconstruct the original image. Second, to achieve better disentanglement, a contrastive loss is proposed to encourage the style codes from the same domain and different domains to be compact and divergent, respectively. Finally, to further improve generalizability, we propose a style augmentation strategy to synthesize images with various unseen styles in real time while maintaining anatomical information. Comprehensive experiments on a public multi-site fundus image dataset and an in-house multi-site Nasopharyngeal Carcinoma Magnetic Resonance Image (NPC-MRI) dataset show that the proposed CDDSA achieved remarkable generalizability across different domains, and it outperformed several state-of-the-art methods in generalizable segmentation. Code is available at https://github.com/HiLab-git/DAG4MIA.

Towards liver segmentation in the wild via contrastive distillation

PURPOSE

Automatic liver segmentation is a key component for performing computer-assisted hepatic procedures. The task is challenging due to the high variability in organ appearance, numerous imaging modalities, and limited availability of labels. Moreover, strong generalization performance is required in real-world scenarios. However, existing supervised methods cannot be applied to data not seen during training (i.e. in the wild) because they generalize poorly.

METHODS

We propose to distill knowledge from a powerful model with our novel contrastive distillation scheme. We use a pre-trained large neural network to train our smaller model. A key novelty is to map neighboring slices close together in the latent representation, while mapping distant slices far away. Then, we use ground-truth labels to learn a U-Net style upsampling path and recover the segmentation map.

RESULTS

The pipeline is proven to be robust enough to perform state-of-the-art inference on target unseen domains. We carried out an extensive experimental validation using six common abdominal datasets, covering multiple modalities, as well as 18 patient datasets from the Innsbruck University Hospital. A sub-second inference time and a data-efficient training pipeline make it possible to scale our method to real-world conditions.

CONCLUSION

We propose a novel contrastive distillation scheme for automatic liver segmentation. A limited set of assumptions and superior performance to state-of-the-art techniques make our method a candidate for application to real-world scenarios.