Rapid artificial intelligence solutions in a pandemic-The COVID-19-20 Lung CT Lesion Segmentation Challenge

Real-world federated learning in radiology: hurdles to overcome and benefits to gain

OBJECTIVE

Federated Learning (FL) enables collaborative model training while keeping data locally. Currently, most FL studies in radiology are conducted in simulated environments due to numerous hurdles impeding its translation into practice. The few existing real-world FL initiatives rarely communicate specific measures taken to overcome these hurdles. To bridge this significant knowledge gap, we propose a comprehensive guide for real-world FL in radiology. Minding efforts to implement real-world FL, there is a lack of comprehensive assessments comparing FL to less complex alternatives in challenging real-world settings, which we address through extensive benchmarking.

MATERIALS AND METHODS

We developed our own FL infrastructure within the German Radiological Cooperative Network (RACOON) and demonstrated its functionality by training FL models on lung pathology segmentation tasks across six university hospitals. Insights gained while establishing our FL initiative and running the extensive benchmark experiments were compiled and categorized into the guide.

RESULTS

The proposed guide outlines essential steps, identified hurdles, and implemented solutions for establishing successful FL initiatives conducting real-world experiments. Our experimental results prove the practical relevance of our guide and show that FL outperforms less complex alternatives in all evaluation scenarios.

DISCUSSION AND CONCLUSION

Our findings justify the efforts required to translate FL into real-world applications by demonstrating advantageous performance over alternative approaches. Additionally, they emphasize the importance of strategic organization, robust management of distributed data and infrastructure in real-world settings. With the proposed guide, we are aiming to aid future FL researchers in circumventing pitfalls and accelerating translation of FL into radiological applications.

Comparative benchmarking of failure detection methods in medical image segmentation: Unveiling the role of confidence aggregation

Semantic segmentation is an essential component of medical image analysis research, with recent deep learning algorithms offering out-of-the-box applicability across diverse datasets. Despite these advancements, segmentation failures remain a significant concern for real-world clinical applications, necessitating reliable detection mechanisms. This paper introduces a comprehensive benchmarking framework aimed at evaluating failure detection methodologies within medical image segmentation. Through our analysis, we identify the strengths and limitations of current failure detection metrics, advocating for the risk-coverage analysis as a holistic evaluation approach. Utilizing a collective dataset comprising five public 3D medical image collections, we assess the efficacy of various failure detection strategies under realistic test-time distribution shifts. Our findings highlight the importance of pixel confidence aggregation and we observe superior performance of the pairwise Dice score (Roy et al., 2019) between ensemble predictions, positioning it as a simple and robust baseline for failure detection in medical image segmentation. To promote ongoing research, we make the benchmarking framework available to the community.

The STOIC2021 COVID-19 AI challenge: Applying reusable training methodologies to private data

Challenges drive the state-of-the-art of automated medical image analysis. The quantity of public training data that they provide can limit the performance of their solutions. Public access to the training methodology for these solutions remains absent. This study implements the Type Three (T3) challenge format, which allows for training solutions on private data and guarantees reusable training methodologies. With T3, challenge organizers train a codebase provided by the participants on sequestered training data. T3 was implemented in the STOIC2021 challenge, with the goal of predicting from a computed tomography (CT) scan whether subjects had a severe COVID-19 infection, defined as intubation or death within one month. STOIC2021 consisted of a Qualification phase, where participants developed challenge solutions using 2000 publicly available CT scans, and a Final phase, where participants submitted their training methodologies with which solutions were trained on CT scans of 9724 subjects. The organizers successfully trained six of the eight Final phase submissions. The submitted codebases for training and running inference were released publicly. The winning solution obtained an area under the receiver operating characteristic curve for discerning between severe and non-severe COVID-19 of 0.815. The Final phase solutions of all finalists improved upon their Qualification phase solutions.

FMD-UNet: fine-grained feature squeeze and multiscale cascade dilated semantic aggregation dual-decoder UNet for COVID-19 lung infection segmentation from CT images

With the advancement of computer-aided diagnosis, the automatic segmentation of COVID-19 infection areas holds great promise for assisting in the timely diagnosis and recovery of patients in clinical practice. Currently, methods relying on U-Net face challenges in effectively utilizing fine-grained semantic information from input images and bridging the semantic gap between the encoder and decoder. To address these issues, we propose an FMD-UNet dual-decoder U-Net network for COVID-19 infection segmentation, which integrates a Fine-grained Feature Squeezing (FGFS) decoder and a Multi-scale Dilated Semantic Aggregation (MDSA) decoder. The FGFS decoder produces fine feature maps through the compression of fine-grained features and a weighted attention mechanism, guiding the model to capture detailed semantic information. The MDSA decoder consists of three hierarchical MDSA modules designed for different stages of input information. These modules progressively fuse different scales of dilated convolutions to process the shallow and deep semantic information from the encoder, and use the extracted feature information to bridge the semantic gaps at various stages, this design captures extensive contextual information while decoding and predicting segmentation, thereby suppressing the increase in model parameters. To better validate the robustness and generalizability of the FMD-UNet, we conducted comprehensive performance evaluations and ablation experiments on three public datasets, and achieved leading Dice Similarity Coefficient (DSC) scores of 84.76, 78.56 and 61.99% in COVID-19 infection segmentation, respectively. Compared to previous methods, the FMD-UNet has fewer parameters and shorter inference time, which also demonstrates its competitiveness.

Lesion region inpainting: an approach for pseudo-healthy image synthesis in intracranial infection imaging

The synthesis of pseudo-healthy images, involving the generation of healthy counterparts for pathological images, is crucial for data augmentation, clinical disease diagnosis, and understanding pathology-induced changes. Recently, Generative Adversarial Networks (GANs) have shown substantial promise in this domain. However, the heterogeneity of intracranial infection symptoms caused by various infections complicates the model's ability to accurately differentiate between pathological and healthy regions, leading to the loss of critical information in healthy areas and impairing the precise preservation of the subject's identity. Moreover, for images with extensive lesion areas, the pseudo-healthy images generated by these methods often lack distinct organ and tissue structures. To address these challenges, we propose a three-stage method (localization, inpainting, synthesis) that achieves nearly perfect preservation of the subject's identity through precise pseudo-healthy synthesis of the lesion region and its surroundings. The process begins with a Segmentor, which identifies the lesion areas and differentiates them from healthy regions. Subsequently, a Vague-Filler fills the lesion areas to construct a healthy outline, thereby preventing structural loss in cases of extensive lesions. Finally, leveraging this healthy outline, a Generative Adversarial Network integrated with a contextual residual attention module generates a more realistic and clearer image. Our method was validated through extensive experiments across different modalities within the BraTS2021 dataset, achieving a healthiness score of 0.957. The visual quality of the generated images markedly exceeded those produced by competing methods, with enhanced capabilities in repairing large lesion areas. Further testing on the COVID-19-20 dataset showed that our model could effectively partially reconstruct images of other organs.

Hybrid Masked Image Modeling for 3D Medical Image Segmentation

Masked image modeling (MIM) with transformer backbones has recently been exploited as a powerful self-supervised pre-training technique. The existing MIM methods adopt the strategy to mask random patches of the image and reconstruct the missing pixels, which only considers semantic information at a lower level, and causes a long pre-training time. This paper presents HybridMIM, a novel hybrid self-supervised learning method based on masked image modeling for 3D medical image segmentation. Specifically, we design a two-level masking hierarchy to specify which and how patches in sub-volumes are masked, effectively providing the constraints of higher level semantic information. Then we learn the semantic information of medical images at three levels, including: 1) partial region prediction to reconstruct key contents of the 3D image, which largely reduces the pre-training time burden (pixel-level); 2) patch-masking perception to learn the spatial relationship between the patches in each sub-volume (region-level); and 3) drop-out-based contrastive learning between samples within a mini-batch, which further improves the generalization ability of the framework (sample-level). The proposed framework is versatile to support both CNN and transformer as encoder backbones, and also enables to pre-train decoders for image segmentation. We conduct comprehensive experiments on five widely-used public medical image segmentation datasets, including BraTS2020, BTCV, MSD Liver, MSD Spleen, and BraTS2023. The experimental results show the clear superiority of HybridMIM against competing supervised methods, masked pre-training approaches, and other self-supervised methods, in terms of quantitative metrics, speed performance and qualitative observations.

LD-UNet: A long-distance perceptual model for segmentation of blurred boundaries in medical images

The blurriness of boundaries in medical image target regions hinders further improvement in automatic segmentation accuracy and is a challenging problem. To address this issue, we propose a model called long-distance perceptual UNet (LD-UNet), which has a powerful long-‍distance perception ability and can effectively perceive the semantic context of an entire image. Specifically, LD-UNet utilizes global and local long-distance induction modules, which endow the model with contextual semantic induction capabilities for long-distance feature dependencies. The modules perform long-distance semantic perception at the high and low stages of LD-UNet, respectively, effectively improving the accuracy of local blurred information assessment. We also propose a top-down deep supervision method to enhance the ability of the model to fit data. Then, extensive experiments on four types of tumor data with blurred boundaries are conducted. The dataset includes nasopharyngeal carcinoma, esophageal carcinoma, pancreatic carcinoma, and colorectal carcinoma. The dice similarity coefficient scores obtained by LD-UNet on the four datasets are 73.35%, 85.93%, 70.04%, and 82.71%. Experimental results demonstrate that LD-UNet is more effective in improving the segmentation accuracy of blurred boundary regions than other methods with long-distance perception, such as transformers. Among all models, LD-UNet achieves the best performance. By visualizing the feature dependency field of the models, we further explore the advantages of LD-UNet in segmenting blurred boundaries.

CT-based Assessment at 6-Month Follow-up of COVID-19 Pneumonia patients in China

This study aimed to assess pulmonary changes at 6-month follow-up CT and predictors of pulmonary residual abnormalities and fibrotic-like changes in COVID-19 pneumonia patients in China following relaxation of COVID restrictions in 2022. A total of 271 hospitalized patients with COVID-19 pneumonia admitted between November 29, 2022 and February 10, 2023 were prospectively evaluated at 6 months. CT characteristics and Chest CT scores of pulmonary abnormalities were compared between the initial and the 6-month CT. The association of demographic and clinical factors with CT residual abnormalities or fibrotic-like changes were assessed using logistic regression. Follow-up CT scans were obtained at a median of 177 days (IQR, 170-185 days) after hospital admission. Pulmonary residual abnormalities and fibrotic-like changes were found in 98 (36.2%) and 39 (14.4%) participants. In multivariable analysis of pulmonary residual abnormalities and fibrotic-like changes, the top three predictive factors were invasive ventilation (OR 13.6; 95% CI 1.9, 45; P < .001), age > 60 years (OR 9.1; 95% CI 2.3, 39; P = .01), paxlovid (OR 0.11; 95% CI 0.04, 0.48; P = .01) and invasive ventilation (OR 10.3; 95% CI 2.9, 33; P = .002), paxlovid (OR 0.1; 95% CI 0.03, 0.48; P = .01), smoker (OR 9.9; 95% CI 2.4, 31; P = .01), respectively. The 6-month follow-up CT of recent COVID-19 pneumonia cases in China showed a considerable proportion of the patients with pulmonary residual abnormalities and fibrotic-like changes. Antivirals against SARS-CoV-2 like paxlovid may be beneficial for long-term regression of COVID-19 pneumonia.

COVID-19 Infection Percentage Estimation from Computed Tomography Scans: Results and Insights from the International Per-COVID-19 Challenge

COVID-19 analysis from medical imaging is an important task that has been intensively studied in the last years due to the spread of the COVID-19 pandemic. In fact, medical imaging has often been used as a complementary or main tool to recognize the infected persons. On the other hand, medical imaging has the ability to provide more details about COVID-19 infection, including its severity and spread, which makes it possible to evaluate the infection and follow-up the patient's state. CT scans are the most informative tool for COVID-19 infection, where the evaluation of COVID-19 infection is usually performed through infection segmentation. However, segmentation is a tedious task that requires much effort and time from expert radiologists. To deal with this limitation, an efficient framework for estimating COVID-19 infection as a regression task is proposed. The goal of the Per-COVID-19 challenge is to test the efficiency of modern deep learning methods on COVID-19 infection percentage estimation (CIPE) from CT scans. Participants had to develop an efficient deep learning approach that can learn from noisy data. In addition, participants had to cope with many challenges, including those related to COVID-19 infection complexity and crossdataset scenarios. This paper provides an overview of the COVID-19 infection percentage estimation challenge (Per-COVID-19) held at MIA-COVID-2022. Details of the competition data, challenges, and evaluation metrics are presented. The best performing approaches and their results are described and discussed.

GIFNet: an effective global infection feature network for automatic COVID-19 lung lesions segmentation

The ongoing COronaVIrus Disease 2019 (COVID-19) pandemic carried by the SARS-CoV-2 virus spread worldwide in early 2019, bringing about an existential health catastrophe. Automatic segmentation of infected lungs from COVID-19 X-ray and computer tomography (CT) images helps to generate a quantitative approach for treatment and diagnosis. The multi-class information about the infected lung is often obtained from the patient's CT dataset. However, the main challenge is the extensive range of infected features and lack of contrast between infected and normal areas. To resolve these issues, a novel Global Infection Feature Network (GIFNet)-based Unet with ResNet50 model is proposed for segmenting the locations of COVID-19 lung infections. The Unet layers have been used to extract the features from input images and select the region of interest (ROI) by using the ResNet50 technique for training it faster. Moreover, integrating the pooling layer into the atrous spatial pyramid pooling (ASPP) mechanism in the bottleneck helps for better feature selection and handles scale variation during training. Furthermore, the partial differential equation (PDE) approach is used to enhance the image quality and intensity value for particular ROI boundary edges in the COVID-19 images. The proposed scheme has been validated on two datasets, namely the SARS-CoV-2 CT scan and COVIDx-19, for detecting infected lung segmentation (ILS). The experimental findings have been subjected to a comprehensive analysis using various evaluation metrics, including accuracy (ACC), area under curve (AUC), recall (REC), specificity (SPE), dice similarity coefficient (DSC), mean absolute error (MAE), precision (PRE), and mean squared error (MSE) to ensure rigorous validation. The results demonstrate the superior performance of the proposed system compared to the state-of-the-art (SOTA) segmentation models on both X-ray and CT datasets.

TestFit: A plug-and-play one-pass test time method for medical image segmentation

Deep learning (DL) based methods have been extensively studied for medical image segmentation, mostly emphasizing the design and training of DL networks. Only few attempts were made on developing methods for applying DL models in test time. In this paper, we study whether a given off-the-shelf segmentation network can be stably improved on-the-fly during test time in an online processing-and-learning fashion. We propose a new online test-time method, called TestFit, to improve results of a given off-the-shelf DL segmentation model in test time by actively fitting the test data distribution. TestFit first creates a supplementary network (SuppNet) from the given trained off-the-shelf segmentation network (this original network is referred to as OGNet) and applies SuppNet together with OGNet for test time inference. OGNet keeps its hypothesis derived from the original training set to prevent the model from collapsing, while SuppNet seeks to fit the test data distribution. Segmentation results and supervision signals (for updating SuppNet) are generated by combining the outputs of OGNet and SuppNet on the fly. TestFit needs only one pass on each test sample - the same as the traditional test model pipeline - and requires no training time preparation. Since it is challenging to look at only one test sample and no manual annotation for model update each time, we develop a series of technical treatments for improving the stability and effectiveness of our proposed online test-time training method. TestFit works in a plug-and-play fashion, requires minimal hyper-parameter tuning, and is easy to use in practice. Experiments on a large collection of 2D and 3D datasets demonstrate the capability of our TestFit method.

Enhancing COVID-19 CT Image Segmentation: A Comparative Study of Attention and Recurrence in UNet Models

Imaging plays a key role in the clinical management of Coronavirus disease 2019 (COVID-19) as the imaging findings reflect the pathological process in the lungs. The visual analysis of High-Resolution Computed Tomography of the chest allows for the differentiation of parenchymal abnormalities of COVID-19, which are crucial to be detected and quantified in order to obtain an accurate disease stratification and prognosis. However, visual assessment and quantification represent a time-consuming task for radiologists. In this regard, tools for semi-automatic segmentation, such as those based on Convolutional Neural Networks, can facilitate the detection of pathological lesions by delineating their contour. In this work, we compared four state-of-the-art Convolutional Neural Networks based on the encoder-decoder paradigm for the binary segmentation of COVID-19 infections after training and testing them on 90 HRCT volumetric scans of patients diagnosed with COVID-19 collected from the database of the Pisa University Hospital. More precisely, we started from a basic model, the well-known UNet, then we added an attention mechanism to obtain an Attention-UNet, and finally we employed a recurrence paradigm to create a Recurrent-Residual UNet (R2-UNet). In the latter case, we also added attention gates to the decoding path of an R2-UNet, thus designing an R2-Attention UNet so as to make the feature representation and accumulation more effective. We compared them to gain understanding of both the cognitive mechanism that can lead a neural model to the best performance for this task and the good compromise between the amount of data, time, and computational resources required. We set up a five-fold cross-validation and assessed the strengths and limitations of these models by evaluating the performances in terms of Dice score, Precision, and Recall defined both on 2D images and on the entire 3D volume. From the results of the analysis, it can be concluded that Attention-UNet outperforms the other models by achieving the best performance of 81.93%, in terms of 2D Dice score, on the test set. Additionally, we conducted statistical analysis to assess the performance differences among the models. Our findings suggest that integrating the recurrence mechanism within the UNet architecture leads to a decline in the model's effectiveness for our particular application.

Robustness of Single- and Dual-Energy Deep-Learning-Based Scatter Correction Models on Simulated and Real Chest X-rays

Chest X-rays (CXRs) represent the first tool globally employed to detect cardiopulmonary pathologies. These acquisitions are highly affected by scattered photons due to the large field of view required. Scatter in CXRs introduces background in the images, which reduces their contrast. We developed three deep-learning-based models to estimate and correct scatter contribution to CXRs. We used a Monte Carlo (MC) ray-tracing model to simulate CXRs from human models obtained from CT scans using different configurations (depending on the availability of dual-energy acquisitions). The simulated CXRs contained the separated contribution of direct and scattered X-rays in the detector. These simulated datasets were then used as the reference for the supervised training of several NNs. Three NN models (single and dual energy) were trained with the MultiResUNet architecture. The performance of the NN models was evaluated on CXRs obtained, with an MC code, from chest CT scans of patients affected by COVID-19. The results show that the NN models were able to estimate and correct the scatter contribution to CXRs with an error of <5%, being robust to variations in the simulation setup and improving contrast in soft tissue. The single-energy model was tested on real CXRs, providing robust estimations of the scatter-corrected CXRs.

Pulmonary contusion: automated deep learning-based quantitative visualization

PURPOSE

Rapid automated CT volumetry of pulmonary contusion may predict progression to Acute Respiratory Distress Syndrome (ARDS) and help guide early clinical management in at-risk trauma patients. This study aims to train and validate state-of-the-art deep learning models to quantify pulmonary contusion as a percentage of total lung volume (Lung Contusion Index, or auto-LCI) and assess the relationship between auto-LCI and relevant clinical outcomes.

METHODS

302 adult patients (age ≥ 18) with pulmonary contusion were retrospectively identified from reports between 2016 and 2021. nnU-Net was trained on manual contusion and whole-lung segmentations. Point-of-care candidate variables for multivariate regression included oxygen saturation, heart rate, and systolic blood pressure on admission. Logistic regression was used to assess ARDS risk, and Cox proportional hazards models were used to determine differences in ICU length of stay and mechanical ventilation time.

RESULTS

Mean Volume Similarity Index and mean Dice scores were 0.82 and 0.67. Interclass correlation coefficient and Pearson r between ground-truth and predicted volumes were 0.90 and 0.91. 38 (14%) patients developed ARDS. In bivariate analysis, auto-LCI was associated with ARDS (p < 0.001), ICU admission (p < 0.001), and need for mechanical ventilation (p < 0.001). In multivariate analyses, auto-LCI was associated with ARDS (p = 0.04), longer length of stay in the ICU (p = 0.02) and longer time on mechanical ventilation (p = 0.04). AUC of multivariate regression to predict ARDS using auto-LCI and clinical variables was 0.70 while AUC using auto-LCI alone was 0.68.

CONCLUSION

Increasing auto-LCI values corresponded with increased risk of ARDS, longer ICU admissions, and longer periods of mechanical ventilation.

Accelerating voxelwise annotation of cross-sectional imaging through AI collaborative labeling with quality assurance and bias mitigation

Background

precision-medicine quantitative tools for cross-sectional imaging require painstaking labeling of targets that vary considerably in volume, prohibiting scaling of data annotation efforts and supervised training to large datasets for robust and generalizable clinical performance. A straight-forward time-saving strategy involves manual editing of AI-generated labels, which we call AI-collaborative labeling (AICL). Factors affecting the efficacy and utility of such an approach are unknown. Reduction in time effort is not well documented. Further, edited AI labels may be prone to automation bias.

Purpose

In this pilot, using a cohort of CTs with intracavitary hemorrhage, we evaluate both time savings and AICL label quality and propose criteria that must be met for using AICL annotations as a high-throughput, high-quality ground truth.

Methods

57 CT scans of patients with traumatic intracavitary hemorrhage were included. No participant recruited for this study had previously interpreted the scans. nnU-net models trained on small existing datasets for each feature (hemothorax/hemoperitoneum/pelvic hematoma; n = 77-253) were used in inference. Two common scenarios served as baseline comparison- de novo expert manual labeling, and expert edits of trained staff labels. Parameters included time effort and image quality graded by a blinded independent expert using a 9-point scale. The observer also attempted to discriminate AICL and expert labels in a random subset (n = 18). Data were compared with ANOVA and post-hoc paired signed rank tests with Bonferroni correction.

Results

AICL reduced time effort 2.8-fold compared to staff label editing, and 8.7-fold compared to expert labeling (corrected p < 0.0006). Mean Likert grades for AICL (8.4, SD:0.6) were significantly higher than for expert labels (7.8, SD:0.9) and edited staff labels (7.7, SD:0.8) (corrected p < 0.0006). The independent observer failed to correctly discriminate AI and human labels.

Conclusion

For our use case and annotators, AICL facilitates rapid large-scale curation of high-quality ground truth. The proposed quality control regime can be employed by other investigators prior to embarking on AICL for segmentation tasks in large datasets.

CT findings in "Post-Covid": residua from acute pneumonia or "Post-Covid-ILD"?

The aim of this study was to evaluate if CT findings in patients with pulmonary Post Covid syndrome represent residua after acute pneumonia or if SARS-CoV 2 induces a true ILD. Consecutive patients with status post acute Covid-19 pneumonia and persisting pulmonary symptoms were enrolled. Inclusion criteria were availability of at least one chest CT performed in the acute phase and at least one chest CT performed at least 80 days after symptom onset. In both acute and chronic phase CTs 14 CT features as well as distribution and extent of opacifications were independently determined by two chest radiologists. Evolution of every single CT lesion over time was registered intraindividually for every patient. Moreover, lung abnormalities were automatically segmented using a pre-trained nnU-Net model and volume as well as density of parenchymal lesions were plotted over the entire course of disease including all available CTs. 29 patients (median age 59 years, IQR 8, 22 men) were enrolled. Follow-up period was 80-242 days (mean 134). 152/157 (97 %) lesions in the chronic phase CTs represented residua of lung pathology in the acute phase. Subjective and objective evaluation of serial CTs showed that CT abnormalities were stable in location and continuously decreasing in extent and density. The results of our study support the hypothesis that CT abnormalities in the chronic phase after Covid-19 pneumonia represent residua in terms of prolonged healing of acute infection. We did not find any evidence for a Post Covid ILD.

PDAtt-Unet: Pyramid Dual-Decoder Attention Unet for Covid-19 infection segmentation from CT-scans

Since the emergence of the Covid-19 pandemic in late 2019, medical imaging has been widely used to analyse this disease. Indeed, CT-scans of the lungs can help diagnose, detect, and quantify Covid-19 infection. In this paper, we address the segmentation of Covid-19 infection from CT-scans. To improve the performance of the Att-Unet architecture and maximize the use of the Attention Gate, we propose the PAtt-Unet and DAtt-Unet architectures. PAtt-Unet aims to exploit the input pyramids to preserve the spatial awareness in all of the encoder layers. On the other hand, DAtt-Unet is designed to guide the segmentation of Covid-19 infection inside the lung lobes. We also propose to combine these two architectures into a single one, which we refer to as PDAtt-Unet. To overcome the blurry boundary pixels segmentation of Covid-19 infection, we propose a hybrid loss function. The proposed architectures were tested on four datasets with two evaluation scenarios (intra and cross datasets). Experimental results showed that both PAtt-Unet and DAtt-Unet improve the performance of Att-Unet in segmenting Covid-19 infections. Moreover, the combination architecture PDAtt-Unet led to further improvement. To Compare with other methods, three baseline segmentation architectures (Unet, Unet++, and Att-Unet) and three state-of-the-art architectures (InfNet, SCOATNet, and nCoVSegNet) were tested. The comparison showed the superiority of the proposed PDAtt-Unet trained with the proposed hybrid loss (PDEAtt-Unet) over all other methods. Moreover, PDEAtt-Unet is able to overcome various challenges in segmenting Covid-19 infections in four datasets and two evaluation scenarios.

Pseudo-Label Guided Image Synthesis for Semi-Supervised COVID-19 Pneumonia Infection Segmentation

Coronavirus disease 2019 (COVID-19) has become a severe global pandemic. Accurate pneumonia infection segmentation is important for assisting doctors in diagnosing COVID-19. Deep learning-based methods can be developed for automatic segmentation, but the lack of large-scale well-annotated COVID-19 training datasets may hinder their performance. Semi-supervised segmentation is a promising solution which explores large amounts of unlabelled data, while most existing methods focus on pseudo-label refinement. In this paper, we propose a new perspective on semi-supervised learning for COVID-19 pneumonia infection segmentation, namely pseudo-label guided image synthesis. The main idea is to keep the pseudo-labels and synthesize new images to match them. The synthetic image has the same COVID-19 infected regions as indicated in the pseudo-label, and the reference style extracted from the style code pool is added to make it more realistic. We introduce two representative methods by incorporating the synthetic images into model training, including single-stage Synthesis-Assisted Cross Pseudo Supervision (SA-CPS) and multi-stage Synthesis-Assisted Self-Training (SA-ST), which can work individually as well as cooperatively. Synthesis-assisted methods expand the training data with high-quality synthetic data, thus improving the segmentation performance. Extensive experiments on two COVID-19 CT datasets for segmenting the infections demonstrate our method is superior to existing schemes for semi-supervised segmentation, and achieves the state-of-the-art performance on both datasets. Code is available at: https://github.com/FeiLyu/SASSL.

CaraNet: context axial reverse attention network for segmentation of small medical objects

Purpose

Segmenting medical images accurately and reliably is important for disease diagnosis and treatment. It is a challenging task because of the wide variety of objects' sizes, shapes, and scanning modalities. Recently, many convolutional neural networks have been designed for segmentation tasks and have achieved great success. Few studies, however, have fully considered the sizes of objects; thus, most demonstrate poor performance for small object segmentation. This can have a significant impact on the early detection of diseases.

Approach

We propose a context axial reverse attention network (CaraNet) to improve the segmentation performance on small objects compared with several recent state-of-the-art models. CaraNet applies axial reserve attention and channel-wise feature pyramid modules to dig the feature information of small medical objects. We evaluate our model by six different measurement metrics.

Results

We test our CaraNet on segmentation datasets for brain tumor (BraTS 2018) and polyp (Kvasir-SEG, CVC-ColonDB, CVC-ClinicDB, CVC-300, and ETIS-LaribPolypDB). Our CaraNet achieves the top-rank mean Dice segmentation accuracy, and results show a distinct advantage of CaraNet in the segmentation of small medical objects.

Conclusions

We proposed CaraNet to segment small medical objects and outperform state-of-the-art methods.

COVID-19 lung infection segmentation from chest CT images based on CAPA-ResUNet

Coronavirus disease 2019 (COVID-19) epidemic has devastating effects on personal health around the world. It is significant to achieve accurate segmentation of pulmonary infection regions, which is an early indicator of disease. To solve this problem, a deep learning model, namely, the content-aware pre-activated residual UNet (CAPA-ResUNet), was proposed for segmenting COVID-19 lesions from CT slices. In this network, the pre-activated residual block was used for down-sampling to solve the problems of complex foreground and large fluctuations of distribution in datasets during training and to avoid gradient disappearance. The area loss function based on the false segmentation regions was proposed to solve the problem of fuzzy boundary of the lesion area. This model was evaluated by the public dataset (COVID-19 Lung CT Lesion Segmentation Challenge-2020) and compared its performance with those of classical models. Our method gains an advantage over other models in multiple metrics. Such as the Dice coefficient, specificity (Spe), and intersection over union (IoU), our CAPA-ResUNet obtained 0.775 points, 0.972 points, and 0.646 points, respectively. The Dice coefficient of our model was 2.51% higher than Content-aware residual UNet (CARes-UNet). The code is available at https://github.com/malu108/LungInfectionSeg.

Domain and Content Adaptive Convolution Based Multi-Source Domain Generalization for Medical Image Segmentation

The domain gap caused mainly by variable medical image quality renders a major obstacle on the path between training a segmentation model in the lab and applying the trained model to unseen clinical data. To address this issue, domain generalization methods have been proposed, which however usually use static convolutions and are less flexible. In this paper, we propose a multi-source domain generalization model based on the domain and content adaptive convolution (DCAC) for the segmentation of medical images across different modalities. Specifically, we design the domain adaptive convolution (DAC) module and content adaptive convolution (CAC) module and incorporate both into an encoder-decoder backbone. In the DAC module, a dynamic convolutional head is conditioned on the predicted domain code of the input to make our model adapt to the unseen target domain. In the CAC module, a dynamic convolutional head is conditioned on the global image features to make our model adapt to the test image. We evaluated the DCAC model against the baseline and four state-of-the-art domain generalization methods on the prostate segmentation, COVID-19 lesion segmentation, and optic cup/optic disc segmentation tasks. Our results not only indicate that the proposed DCAC model outperforms all competing methods on each segmentation task but also demonstrate the effectiveness of the DAC and CAC modules. Code is available at https://git.io/DCAC.

Computed tomography-based COVID-19 triage through a deep neural network using mask-weighted global average pooling

Background

There is an urgent need to find an effective and accurate method for triaging coronavirus disease 2019 (COVID-19) patients from millions or billions of people. Therefore, this study aimed to develop a novel deep-learning approach for COVID-19 triage based on chest computed tomography (CT) images, including normal, pneumonia, and COVID-19 cases.

Methods

A total of 2,809 chest CT scans (1,105 COVID-19, 854 normal, and 850 non-3COVID-19 pneumonia cases) were acquired for this study and classified into the training set (n = 2,329) and test set (n = 480). A U-net-based convolutional neural network was used for lung segmentation, and a mask-weighted global average pooling (GAP) method was proposed for the deep neural network to improve the performance of COVID-19 classification between COVID-19 and normal or common pneumonia cases.

Results

The results for lung segmentation reached a dice value of 96.5% on 30 independent CT scans. The performance of the mask-weighted GAP method achieved the COVID-19 triage with a sensitivity of 96.5% and specificity of 87.8% using the testing dataset. The mask-weighted GAP method demonstrated 0.9% and 2% improvements in sensitivity and specificity, respectively, compared with the normal GAP. In addition, fusion images between the CT images and the highlighted area from the deep learning model using the Grad-CAM method, indicating the lesion region detected using the deep learning method, were drawn and could also be confirmed by radiologists.

Conclusions

This study proposed a mask-weighted GAP-based deep learning method and obtained promising results for COVID-19 triage based on chest CT images. Furthermore, it can be considered a convenient tool to assist doctors in diagnosing COVID-19.

SR-CycleGAN: super-resolution of clinical CT to micro-CT level with multi-modality super-resolution loss

Purpose: We propose a super-resolution (SR) method, named SR-CycleGAN, for SR of clinical computed tomography (CT) images to the micro-focus x-ray CT CT ( μ CT ) level. Due to the resolution limitations of clinical CT (about 500 × 500 × 500    μ m 3 / voxel ), it is challenging to obtain enough pathological information. On the other hand, μ CT scanning allows the imaging of lung specimens with significantly higher resolution (about 50 × 50 × 50    μ m 3 / voxel or higher), which allows us to obtain and analyze detailed anatomical information. As a way to obtain detailed information such as cancer invasion and bronchioles from preoperative clinical CT images of lung cancer patients, the SR of clinical CT images to the μ CT level is desired. Approach: Typical SR methods require aligned pairs of low-resolution (LR) and high-resolution images for training, but it is infeasible to obtain precisely aligned paired clinical CT and μ CT images. To solve this problem, we propose an unpaired SR approach that can perform SR on clinical CT to the μ CT level. We modify a conventional image-to-image translation network named CycleGAN to an inter-modality translation network named SR-CycleGAN. The modifications consist of three parts: (1) an innovative loss function named multi-modality super-resolution loss, (2) optimized SR network structures for enlarging the input LR image to2 k -times by width and height to obtain the SR output, and (3) sub-pixel shuffling layers for reducing computing time. Results: Experimental results demonstrated that our method successfully performed SR of lung clinical CT images. SSIM and PSNR scores of our method were 0.54 and 17.71, higher than the conventional CycleGAN's scores of 0.05 and 13.64, respectively. Conclusions: The proposed SR-CycleGAN is usable for the SR of a lung clinical CT into μ CT scale, while conventional CycleGAN output images with low qualitative and quantitative values. More lung micro-anatomy information could be observed to aid diagnosis, such as the shape of bronchioles walls.