High-Resolution Chest X-Ray Bone Suppression Using Unpaired CT Structural Priors

PELE scores: pelvic X-ray landmark detection with pelvis extraction and enhancement

PURPOSE

Pelvic X-ray (PXR) is widely utilized in clinical decision-making associated with the pelvis, the lower part of the trunk that supports and balances the trunk. In particular, PXR-based landmark detection facilitates downstream analysis and computer-assisted diagnosis and treatment of pelvic diseases. Although PXR has the advantages of low radiation and reduced cost compared to computed tomography (CT), it characterizes the 2D pelvis-tissue superposition of 3D structures, which may affect the accuracy of landmark detection in some cases. However, the superposition nature of PXR is implicitly handled by existing deep learning-based landmark detection methods, which mainly design the deep network structures for better detection performances. Explicit handling of the superposition nature of PXR is rarely done.

METHODS

In this paper, we explicitly focus on the superposition of X-ray images. Specifically, we propose a pelvis extraction (PELE) module that consists of a decomposition network, a domain adaptation network, and an enhancement module, which utilizes 3D prior anatomical knowledge in CT to guide and well isolate the pelvis from PXR, thereby eliminating the influence of soft tissue for landmark detection. The extracted pelvis image, after enhancement, is then used for landmark detection.

RESULTS

We conduct an extensive evaluation based on two public and one private dataset, totaling 850 PXRs. The experimental results show that the proposed PELE module significantly improves the accuracy of PXRs landmark detection and achieves state-of-the-art performances in several benchmark metrics.

CONCLUSION

The design of PELE module can improve the accuracy of different pelvic landmark detection baselines, which we believe is obviously conducive to the positioning and inspection of clinical landmarks and critical structures, thus better serving downstream tasks. Our project has been open-sourced at https://github.com/ECNUACRush/PELEscores .

Decomposition of musculoskeletal structures from radiographs using an improved CycleGAN framework

This paper presents methods of decomposition of musculoskeletal structures from radiographs into multiple individual muscle and bone structures. While existing solutions require dual-energy scan for the training dataset and are mainly applied to structures with high-intensity contrast, such as bones, we focused on multiple superimposed muscles with subtle contrast in addition to bones. The decomposition problem is formulated as an image translation problem between (1) a real X-ray image and (2) multiple digitally reconstructed radiographs, each of which contains a single muscle or bone structure, and solved using unpaired training based on the CycleGAN framework. The training dataset was created via automatic computed tomography (CT) segmentation of muscle/bone regions and virtually projecting them with geometric parameters similar to the real X-ray images. Two additional features were incorporated into the CycleGAN framework to achieve a high-resolution and accurate decomposition: hierarchical learning and reconstruction loss with the gradient correlation similarity metric. Furthermore, we introduced a new diagnostic metric for muscle asymmetry directly measured from a plain X-ray image to validate the proposed method. Our simulation and real-image experiments using real X-ray and CT images of 475 patients with hip diseases suggested that each additional feature significantly enhanced the decomposition accuracy. The experiments also evaluated the accuracy of muscle volume ratio measurement, which suggested a potential application to muscle asymmetry assessment from an X-ray image for diagnostic and therapeutic assistance. The improved CycleGAN framework can be applied for investigating the decomposition of musculoskeletal structures from single radiographs.

Unpaired, unsupervised domain adaptation assumes your domains are already similar

Unsupervised domain adaptation is a popular method in medical image analysis, but it can be tricky to make it work: without labels to link the domains, domains must be matched using feature distributions. If there is no additional information, this often leaves a choice between multiple possibilities to map the data that may be equally likely but not equally correct. In this paper we explore the fundamental problems that may arise in unsupervised domain adaptation, and discuss conditions that might still make it work. Focusing on medical image analysis, we argue that images from different domains may have similar class balance, similar intensities, similar spatial structure, or similar textures. We demonstrate how these implicit conditions can affect domain adaptation performance in experiments with synthetic data, MNIST digits, and medical images. We observe that practical success of unsupervised domain adaptation relies on existing similarities in the data, and is anything but guaranteed in the general case. Understanding these implicit assumptions is a key step in identifying potential problems in domain adaptation and improving the reliability of the results.

Bone suppression of lateral chest x-rays with imperfect and limited dual-energy subtraction images

Bone suppression is to suppress the superimposed bone components over the soft tissues within the lung area of Chest X-ray (CXR), which is potentially useful for the subsequent lung disease diagnosis for radiologists, as well as computer-aided systems. Despite bone suppression methods for frontal CXRs being well studied, it remains challenging for lateral CXRs due to the limited and imperfect DES dataset containing paired lateral CXR and soft-tissue/bone images and more complex anatomical structures in the lateral view. In this work, we propose a bone suppression method for lateral CXRs by leveraging a two-stage distillation learning strategy and a specific data correction method. Specifically, a primary model is first trained on a real DES dataset with limited samples. The bone-suppressed results on a relatively large lateral CXR dataset produced by the primary model are improved by a designed gradient correction method. Secondly, the corrected results serve as training samples to train the distillated model. By automatically learning knowledge from both the primary model and the extra correction procedure, our distillated model is expected to promote the performance of the primary model while omitting the tedious correction procedure. We adopt an ensemble model named MsDd-MAP for the primary and distillated models, which learns the complementary information of Multi-scale and Dual-domain (i.e., intensity and gradient) and fuses them in a maximum-a-posteriori (MAP) framework. Our method is evaluated on a two-exposure lateral DES dataset consisting of 46 subjects and a lateral CXR dataset consisting of 240 subjects. The experimental results suggest that our method is superior to other competing methods regarding the quantitative evaluation metrics. Furthermore, the subjective evaluation by three experienced radiologists also indicates that the distillated model can produce more visually appealing soft-tissue images than the primary model, even comparable to real DES imaging for lateral CXRs.

Assessing the Impact of Image Resolution on Deep Learning for TB Lesion Segmentation on Frontal Chest X-rays

Deep learning (DL) models are state-of-the-art in segmenting anatomical and disease regions of interest (ROIs) in medical images. Particularly, a large number of DL-based techniques have been reported using chest X-rays (CXRs). However, these models are reportedly trained on reduced image resolutions for reasons related to the lack of computational resources. Literature is sparse in discussing the optimal image resolution to train these models for segmenting the tuberculosis (TB)-consistent lesions in CXRs. In this study, we investigated the performance variations with an Inception-V3 UNet model using various image resolutions with/without lung ROI cropping and aspect ratio adjustments and identified the optimal image resolution through extensive empirical evaluations to improve TB-consistent lesion segmentation performance. We used the Shenzhen CXR dataset for the study, which includes 326 normal patients and 336 TB patients. We proposed a combinatorial approach consisting of storing model snapshots, optimizing segmentation threshold and test-time augmentation (TTA), and averaging the snapshot predictions, to further improve performance with the optimal resolution. Our experimental results demonstrate that higher image resolutions are not always necessary; however, identifying the optimal image resolution is critical to achieving superior performance.

Pairwise attention-enhanced adversarial model for automatic bone segmentation in CT images

Objective. Bone segmentation is a critical step in screw placement navigation. Although the deep learning methods have promoted the rapid development for bone segmentation, the local bone separation is still challenging due to irregular shapes and similar representational features.Approach. In this paper, we proposed the pairwise attention-enhanced adversarial model (Pair-SegAM) for automatic bone segmentation in computed tomography images, which includes the two parts of the segmentation model and discriminator. Considering that the distributions of the predictions from the segmentation model contains complicated semantics, we improve the discriminator to strengthen the awareness ability of the target region, improving the parsing of semantic information features. The Pair-SegAM has a pairwise structure, which uses two calculation mechanics to set up pairwise attention maps, then we utilize the semantic fusion to filter unstable regions. Therefore, the improved discriminator provides more refinement information to capture the bone outline, thus effectively enhancing the segmentation models for bone segmentation.Main results. To test the Pair-SegAM, we selected the two bone datasets for assessment. We evaluated our method against several bone segmentation models and latest adversarial models on the both datasets. The experimental results prove that our method not only exhibits superior bone segmentation performance, but also states effective generalization.Significance. Our method provides a more efficient segmentation of specific bones and has the potential to be extended to other semantic segmentation domains.

Prediction of Individual Lymph Node Metastatic Status in Esophageal Squamous Cell Carcinoma Using Routine Computed Tomography Imaging: Comparison of Size-Based Measurements and Radiomics-Based Models

BACKGROUND

Lymph node status is vital for prognosis and treatment decisions for esophageal squamous cell carcinoma (ESCC). This study aimed to construct and evaluate an optimal radiomics-based method for a more accurate evaluation of individual regional lymph node status in ESCC and to compare it with traditional size-based measurements.

METHODS

The study consecutively collected 3225 regional lymph nodes from 530 ESCC patients receiving upfront surgery from January 2011 to October 2015. Computed tomography (CT) scans for individual lymph nodes were analyzed. The study evaluated the predictive performance of machine-learning models trained on features extracted from two-dimensional (2D) and three-dimensional (3D) radiomics by different contouring methods. Robust and important radiomics features were selected, and classification models were further established and validated.

RESULTS

The lymph node metastasis rate was 13.2% (427/3225). The average short-axis diameter was 6.4 mm for benign lymph nodes and 7.9 mm for metastatic lymph nodes. The division of lymph node stations into five regions according to anatomic lymph node drainage (cervical, upper mediastinal, middle mediastinal, lower mediastinal, and abdominal regions) improved the predictive performance. The 2D radiomics method showed optimal diagnostic results, with more efficient segmentation of nodal lesions. In the test set, this optimal model achieved an area under the receiver operating characteristic curve of 0.841-0.891, an accuracy of 84.2-94.7%, a sensitivity of 65.7-83.3%, and a specificity of 84.4-96.7%.

CONCLUSIONS

The 2D radiomics-based models noninvasively predicted the metastatic status of an individual lymph node in ESCC and outperformed the conventional size-based measurement. The 2D radiomics-based model could be incorporated into the current clinical workflow to enable better decision-making for treatment strategies.

Spatial feature and resolution maximization GAN for bone suppression in chest radiographs

BACKGROUND AND OBJECTIVE

Chest radiographs (CXR) are in great demand for visualizing the pathology of the lungs. However, the appearance of bones in the lung region hinders the localization of any lesion or nodule present in the CXR. Thus, bone suppression becomes an important task for the effective screening of lung diseases. Simultaneously, it is equally important to preserve spatial information and image quality because they provide crucial insights on the size and area of infection, color accuracy, structural quality, etc. Many researchers considered bone suppression as an image denoising problem and proposed conditional Generative Adversarial Network-based (cGAN) models for generating bone suppressed images from CXRs. These works do not focus on the retention of spatial features and image quality. The authors of this manuscript developed the Spatial Feature and Resolution Maximization (SFRM) GAN to efficiently minimize the visibility of bones in CXRs while ensuring maximum retention of critical information.

METHOD

This task is achieved by modifying the architectures of the discriminator and generator of the pix2pix model. The discriminator is combined with the Wasserstein GAN with Gradient Penalty to increase its performance and training stability. For the generator, a combination of different task-specific loss functions, viz., L1, Perceptual, and Sobel loss are employed to capture the intrinsic information in the image.

RESULT

The proposed model reported as measures of performance a mean PSNR of 43.588, mean NMSE of 0.00025, mean SSIM of 0.989, and mean Entropy of 0.454 bits/pixel on a test size of 100 images. Further, the combination of δ=10⁴, α=1, β=10, and γ=10 are the hyperparameters that provided the best trade-off between image denoising and quality retention.

CONCLUSION

The degree of bone suppression and spatial information preservation can be improved by adding the Sobel and Perceptual loss respectively. SFRM-GAN not only suppresses bones but also retains the image quality and intrinsic information. Based on the results of student's t-test it is concluded that SFRM-GAN yields statistically significant results at a 0.95 level of confidence and shows its supremacy over the state-of-the-art models. Thus, it may be used for denoising and preprocessing of images.

Semantic segmentation of bone structures in chest X-rays including unhealthy radiographs: A robust and accurate approach

The chest X-ray is a widely used medical imaging technique for the diagnosis of several lung diseases. Some nodules or other pathologies present in the lungs are difficult to visualize on chest X-rays because they are obscured byoverlying boneshadows. Segmentation of bone structures and suppressing them assist medical professionals in reliable diagnosis and organ morphometry. But segmentation of bone structures is challenging due to fuzzy boundaries of organs and inconsistent shape and size of organs due to health issues, age, and gender. The existing bone segmentation methods do not report their performance on abnormal chest X-rays, where it is even more critical to segment the bones. This work presents a robust encoder-decoder network for semantic segmentation of bone structures on normal as well as abnormal chest X-rays. The novelty here lies in combining techniques from two existing networks (Deeplabv3+ and U-net) to achieve robust and superior performance. The fully connected layers of the pre-trained ResNet50 network have been replaced by an Atrous spatial pyramid pooling block for improving the quality of the embedding in the encoder module. The decoder module includes four times upsampling blocks to connect both low-level and high-level features information enabling us to retain both the edges and detail information of the objects. At each level, the up-sampled decoder features are concatenated with the encoder features at a similar level and further fine-tuned to refine the segmentation output. We construct a diverse chest X-ray dataset with ground truth binary masks of anterior ribs, posterior ribs, and clavicle bone for experimentation. The dataset includes 100 samples of chest X-rays belonging to healthy and confirmed patients of lung diseases to maintain the diversity and test the robustness of our method. We test our method using multiple standard metrics and experimental results indicate an excellent performance on both normal and abnormal chest X-rays.

Bone suppression on pediatric chest radiographs via a deep learning-based cascade model

BACKGROUND AND OBJECTIVE

Bone suppression images (BSIs) of chest radiographs (CXRs) have been proven to improve diagnosis of pulmonary diseases. To acquire BSIs, dual-energy subtraction (DES) or a deep-learning-based model trained with DES-based BSIs have been used. However, neither technique could be applied to pediatric patients owing to the harmful effects of DES. In this study, we developed a novel method for bone suppression in pediatric CXRs.

METHODS

First, a model using digitally reconstructed radiographs (DRRs) of adults, which were used to generate pseudo-CXRs from computed tomography images, was developed by training a 2-channel contrastive-unpaired-image-translation network. Second, this model was applied to 129 pediatric DRRs to generate the paired training data of pseudo-pediatric CXRs. Finally, by training a U-Net with these paired data, a bone suppression model for pediatric CXRs was developed.

RESULTS

The evaluation metrics were peak signal to noise ratio, root mean absolute error and structural similarity index measure at soft-tissue and bone region of the lung. In addition, an expert radiologist scored the effectiveness of BSIs on a scale of 1-5. The obtained result of 3.31 ± 0.48 indicates that the BSIs show homogeneous bone removal despite subtle residual bone shadow.

CONCLUSION

Our method shows that the pixel intensity at soft-tissue regions was preserved, and bones were well subtracted; this can be useful for detecting early pulmonary disease in pediatric CXRs.

Pathogeny Detection for Mild Cognitive Impairment via Weighted Evolutionary Random Forest with Brain Imaging and Genetic Data

Medical imaging technology and gene sequencing technology have long been widely used to analyze the pathogenesis and make precise diagnoses of mild cognitive impairment (MCI). However, few studies involve the fusion of radiomics data with genomics data to make full use of the complementarity between different omics to detect pathogenic factors of MCI. This paper performs multimodal fusion analysis based on functional magnetic resonance imaging (fMRI) data and single nucleotide polymorphism (SNP) data of MCI patients. In specific, first, using correlation analysis methods on sequence information of regions of interests (ROIs) and digitalized gene sequences, the fusion features of samples are constructed. Then, introducing weighted evolution strategy into ensemble learning, a novel weighted evolutionary random forest (WERF) model is built to eliminate the inefficient features. Consequently, with the help of WERF, an overall multimodal data analysis framework is established to effectively identify MCI patients and extract pathogenic factors. Based on the data of MCI patients from the ADNI database and compared with some existing popular methods, the superiority in performance of the framework is verified. Our study has great potential to be an effective tool for pathogenic factors detection of MCI.

Chest X-ray Bone Suppression for Improving Classification of Tuberculosis-Consistent Findings

Chest X-rays (CXRs) are the most commonly performed diagnostic examination to detect cardiopulmonary abnormalities. However, the presence of bony structures such as ribs and clavicles can obscure subtle abnormalities, resulting in diagnostic errors. This study aims to build a deep learning (DL)-based bone suppression model that identifies and removes these occluding bony structures in frontal CXRs to assist in reducing errors in radiological interpretation, including DL workflows, related to detecting manifestations consistent with tuberculosis (TB). Several bone suppression models with various deep architectures are trained and optimized using the proposed combined loss function and their performances are evaluated in a cross-institutional test setting using several metrics such as mean absolute error (MAE), peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and multiscale structural similarity measure (MS-SSIM). The best-performing model (ResNet-BS) (PSNR = 34.0678; MS-SSIM = 0.9828) is used to suppress bones in the publicly available Shenzhen and Montgomery TB CXR collections. A VGG-16 model is pretrained on a large collection of publicly available CXRs. The CXR-pretrained model is then fine-tuned individually on the non-bone-suppressed and bone-suppressed CXRs of Shenzhen and Montgomery TB CXR collections to classify them as showing normal lungs or TB manifestations. The performances of these models are compared using several performance metrics such as accuracy, the area under the curve (AUC), sensitivity, specificity, precision, F-score, and Matthews correlation coefficient (MCC), analyzed for statistical significance, and their predictions are qualitatively interpreted through class-selective relevance maps (CRMs). It is observed that the models trained on bone-suppressed CXRs (Shenzhen: AUC = 0.9535 ± 0.0186; Montgomery: AUC = 0.9635 ± 0.0106) significantly outperformed (p < 0.05) the models trained on the non-bone-suppressed CXRs (Shenzhen: AUC = 0.8991 ± 0.0268; Montgomery: AUC = 0.8567 ± 0.0870).. Models trained on bone-suppressed CXRs improved detection of TB-consistent findings and resulted in compact clustering of the data points in the feature space signifying that bone suppression improved the model sensitivity toward TB classification.

A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises

Since its renaissance, deep learning has been widely used in various medical imaging tasks and has achieved remarkable success in many medical imaging applications, thereby propelling us into the so-called artificial intelligence (AI) era. It is known that the success of AI is mostly attributed to the availability of big data with annotations for a single task and the advances in high performance computing. However, medical imaging presents unique challenges that confront deep learning approaches. In this survey paper, we first present traits of medical imaging, highlight both clinical needs and technical challenges in medical imaging, and describe how emerging trends in deep learning are addressing these issues. We cover the topics of network architecture, sparse and noisy labels, federating learning, interpretability, uncertainty quantification, etc. Then, we present several case studies that are commonly found in clinical practice, including digital pathology and chest, brain, cardiovascular, and abdominal imaging. Rather than presenting an exhaustive literature survey, we instead describe some prominent research highlights related to these case study applications. We conclude with a discussion and presentation of promising future directions.

Contrast-Attentive Thoracic Disease Recognition With Dual-Weighting Graph Reasoning

Automatic thoracic disease diagnosis is a rising research topic in the medical imaging community, with many potential applications. However, the inconsistent appearances and high complexities of various lesions in chest X-rays currently hinder the development of a reliable and robust intelligent diagnosis system. Attending to the high-probability abnormal regions and exploiting the priori of a related knowledge graph offers one promising route to addressing these issues. As such, in this paper, we propose two contrastive abnormal attention models and a dual-weighting graph convolution to improve the performance of thoracic multi-disease recognition. First, a left-right lung contrastive network is designed to learn intra-attentive abnormal features to better identify the most common thoracic diseases, whose lesions rarely appear in both sides symmetrically. Moreover, an inter-contrastive abnormal attention model aims to compare the query scan with multiple anchor scans without lesions to compute the abnormal attention map. Once the intra- and inter-contrastive attentions are weighted over the features, in addition to the basic visual spatial convolution, a chest radiology graph is constructed for dual-weighting graph reasoning. Extensive experiments on the public NIH ChestX-ray and CheXpert datasets show that our model achieves consistent improvements over the state-of-the-art methods both on thoracic disease identification and localization.