A new paradigm of interactive artery/vein separation in noncontrast pulmonary CT imaging using multiscale topomorphologic opening

MSO-GP: 3-D segmentation of large and complex conjoined tree structures

Robust segmentation of large and complex conjoined tree structures in 3-D is a major challenge in computer vision. This is particularly true in computational biology, where we often encounter large data structures in size, but few in number, which poses a hard problem for learning algorithms. We show that merging multiscale opening with geodesic path propagation, can shed new light on this classic machine vision challenge, while circumventing the learning issue by developing an unsupervised visual geometry approach (digital topology/morphometry). The novelty of the proposed MSO-GP method comes from the geodesic path propagation being guided by a skeletonization of the conjoined structure that helps to achieve robust segmentation results in a particularly challenging task in this area, that of artery-vein separation from non-contrast pulmonary computed tomography angiograms. This is an important first step in measuring vascular geometry to then diagnose pulmonary diseases and to develop image-based phenotypes. We first present proof-of-concept results on synthetic data, and then verify the performance on pig lung and human lung data with less segmentation time and user intervention needs than those of the competing methods.

A new segment method for pulmonary artery and vein

Accurate differentiation between pulmonary arteries and veins (A/V) holds pivotal importance in the realm of diagnosing and treating pulmonary ailments. This study presents a new approach that leverages grayscale differences between A/V. Distinctions are measured using median and mean grayscale values within the vessel area. Initially, adherent regions are removed based on vessel structure. The trunk regions are segmented using gray level information near the heart region of the lung boundary. Incorrectly segmented vessels are corrected based on connectivity. For distal lung vessels, a similar distance field is established using a graph-cut method. Experimental results show the algorithm's superior segmentation accuracy, achieving 97.26% compared to the CNN-based average accuracy of 91.67%. Error branches are more concentrated, aiding subsequent manual and automatic correction. This demonstrates the algorithm's effective segmentation of pulmonary A/V.

Automated detection and segmentation of pulmonary embolisms on computed tomography pulmonary angiography (CTPA) using deep learning but without manual outlining

We present a novel computer algorithm to automatically detect and segment pulmonary embolisms (PEs) on computed tomography pulmonary angiography (CTPA). This algorithm is based on deep learning but does not require manual outlines of the PE regions. Given a CTPA scan, both intra- and extra-pulmonary arteries were firstly segmented. The arteries were then partitioned into several parts based on size (radius). Adaptive thresholding and constrained morphological operations were used to identify suspicious PE regions within each part. The confidence of a suspicious region to be PE was scored based on its contrast in the arteries. This approach was applied to the publicly available RSNA Pulmonary Embolism CT Dataset (RSNA-PE) to identify three-dimensional (3-D) PE negative and positive image patches, which were used to train a 3-D Recurrent Residual U-Net (R2-Unet) to automatically segment PE. The feasibility of this computer algorithm was validated on an independent test set consisting of 91 CTPA scans acquired from a different medical institute, where the PE regions were manually located and outlined by a thoracic radiologist (>18 years' experience). An R2-Unet model was also trained and validated on the manual outlines using a 5-fold cross-validation method. The CNN model trained on the high-confident PE regions showed a Dice coefficient of 0.676±0.168 and a false positive rate of 1.86 per CT scan, while the CNN model trained on the manual outlines demonstrated a Dice coefficient of 0.647±0.192 and a false positive rate of 4.20 per CT scan. The former model performed significantly better than the latter model (p<0.01). The promising performance of the developed PE detection and segmentation algorithm suggests the feasibility of training a deep learning network without dedicating significant efforts to manual annotations of the PE regions on CTPA scans.

Learning multi-view and centerline topology connectivity information for pulmonary artery-vein separation

BACKGROUND

Automatic pulmonary artery-vein separation has considerable importance in the diagnosis and treatment of lung diseases. However, insufficient connectivity and spatial inconsistency have always been the problems of artery-vein separation.

METHODS

A novel automatic method for artery-vein separation in CT images is presented in this work. Specifically, a multi-scale information aggregated network (MSIA-Net) including multi-scale fusion blocks and deep supervision, is proposed to learn the features of artery-vein and aggregate additional semantic information, respectively. The proposed method integrates nine MSIA-Net models for artery-vein separation, vessel segmentation, and centerline separation tasks along with axial, coronal, and sagittal multi-view slices. First, the preliminary artery-vein separation results are obtained by the proposed multi-view fusion strategy (MVFS). Then, centerline correction algorithm (CCA) is used to correct the preliminary results of artery-vein separation by the centerline separation results. Finally, the vessel segmentation results are utilized to reconstruct the artery-vein morphology. In addition, weighted cross-entropy and dice loss are employed to solve the class imbalance problem.

RESULTS

We constructed 50 manually labeled contrast-enhanced computed CT scans for five-fold cross-validation, and experimental results demonstrated that our method achieves superior segmentation performance of 97.7%, 85.1%, and 84.9% on ACC, Pre, and DSC, respectively. Additionally, a series of ablation studies demonstrate the effectiveness of the proposed components.

CONCLUSION

The proposed method can effectively solve the problem of insufficient vascular connectivity and correct the spatial inconsistency of artery-vein.

Automated identification of pulmonary arteries and veins depicted in non-contrast chest CT scans

We present a novel integrative computerized solution to automatically identify and differentiate pulmonary arteries and veins depicted on chest computed tomography (CT) without iodinated contrast agents. We first identified the central extrapulmonary arteries and veins using a convolutional neural network (CNN) model. Then, a computational differential geometry method was used to automatically identify the tubular-like structures in the lungs with high densities, which we believe are the intrapulmonary vessels. Beginning with the extrapulmonary arteries and veins, we progressively traced the intrapulmonary vessels by following their skeletons and differentiated them into arteries and veins. Instead of manually labeling the numerous arteries and veins in the lungs for machine learning, this integrative strategy limits the manual effort only to the large extrapulmonary vessels. We used a dataset consisting of 120 chest CT scans acquired on different subjects using various protocols to develop, train, and test the algorithms. Our experiments on an independent test set (n = 15) showed promising performance. The computer algorithm achieved a sensitivity of ∼98% in labeling the pulmonary artery and vein branches when compared with a human expert's results, demonstrating the feasibility of our computerized solution in pulmonary artery/vein labeling.

A Pulmonary Artery-Vein Separation Algorithm Based on the Relationship between Subtrees Information

Physicians need to distinguish between pulmonary arteries and veins when diagnosing diseases such as chronic obstructive pulmonary disease (COPD) and lung tumors. However, manual differentiation is difficult due to various factors such as equipment and body structure. Unlike previous geometric methods of manually selecting the points of seeds and using neural networks for separation, this paper proposes a combined algorithm for pulmonary artery-vein separation based on subtree relationship by implementing a completely new idea and combining global and local information, anatomical knowledge, and two-dimensional region growing method. The algorithm completes the reconstruction of the whole vascular structure and the separation of adhesion points from the tree-like structure characteristics of blood vessels, after which the automatic classification of arteries and veins is achieved by using anatomical knowledge, and the whole process is free from human intervention. After comparing all the experimental results with the gold standard, we obtained an average separation accuracy of 85%, which achieved effective separation. Meanwhile, the time range could be controlled between 40 s and 50 s, indicating that the algorithm has good stability.

Learning Tubule-Sensitive CNNs for Pulmonary Airway and Artery-Vein Segmentation in CT

Training convolutional neural networks (CNNs) for segmentation of pulmonary airway, artery, and vein is challenging due to sparse supervisory signals caused by the severe class imbalance between tubular targets and background. We present a CNNs-based method for accurate airway and artery-vein segmentation in non-contrast computed tomography. It enjoys superior sensitivity to tenuous peripheral bronchioles, arterioles, and venules. The method first uses a feature recalibration module to make the best use of features learned from the neural networks. Spatial information of features is properly integrated to retain relative priority of activated regions, which benefits the subsequent channel-wise recalibration. Then, attention distillation module is introduced to reinforce representation learning of tubular objects. Fine-grained details in high-resolution attention maps are passing down from one layer to its previous layer recursively to enrich context. Anatomy prior of lung context map and distance transform map is designed and incorporated for better artery-vein differentiation capacity. Extensive experiments demonstrated considerable performance gains brought by these components. Compared with state-of-the-art methods, our method extracted much more branches while maintaining competitive overall segmentation performance. Codes and models are available at http://www.pami.sjtu.edu.cn/News/56.

A graph-cut approach for pulmonary artery-vein segmentation in noncontrast CT images

Lung vessel segmentation has been widely explored by the biomedical image processing community; however, the differentiation of arterial from venous irrigation is still a challenge. Pulmonary artery-vein (AV) segmentation using computed tomography (CT) is growing in importance owing to its undeniable utility in multiple cardiopulmonary pathological states, especially those implying vascular remodelling, allowing the study of both flow systems separately. We present a new framework to approach the separation of tree-like structures using local information and a specifically designed graph-cut methodology that ensures connectivity as well as the spatial and directional consistency of the derived subtrees. This framework has been applied to the pulmonary AV classification using a random forest (RF) pre-classifier to exploit the local anatomical differences of arteries and veins. The evaluation of the system was performed using 192 bronchopulmonary segment phantoms, 48 anthropomorphic pulmonary CT phantoms, and 26 lungs from noncontrast CT images with precise voxel-based reference standards obtained by manually labelling the vessel trees. The experiments reveal a relevant improvement in the accuracy ( ∼ 20%) of the vessel particle classification with the proposed framework with respect to using only the pre-classification based on local information applied to the whole area of the lung under study. The results demonstrated the accurate differentiation between arteries and veins in both clinical and synthetic cases, specifically when the image quality can guarantee a good airway segmentation, which opens a huge range of possibilities in the clinical study of cardiopulmonary diseases.

Pulmonary Artery-Vein Classification in CT Images Using Deep Learning

Recent studies show that pulmonary vascular diseases may specifically affect arteries or veins through different physiologic mechanisms. To detect changes in the two vascular trees, physicians manually analyze the chest computed tomography (CT) image of the patients in search of abnormalities. This process is time consuming, difficult to standardize, and thus not feasible for large clinical studies or useful in real-world clinical decision making. Therefore, automatic separation of arteries and veins in CT images is becoming of great interest, as it may help physicians to accurately diagnose pathological conditions. In this paper, we present a novel, fully automatic approach to classify vessels from chest CT images into arteries and veins. The algorithm follows three main steps: first, a scale-space particles segmentation to isolate vessels; then a 3-D convolutional neural network (CNN) to obtain a first classification of vessels; finally, graph-cuts' optimization to refine the results. To justify the usage of the proposed CNN architecture, we compared different 2-D and 3-D CNNs that may use local information from bronchus- and vessel-enhanced images provided to the network with different strategies. We also compared the proposed CNN approach with a random forests (RFs) classifier. The methodology was trained and evaluated on the superior and inferior lobes of the right lung of 18 clinical cases with noncontrast chest CT scans, in comparison with manual classification. The proposed algorithm achieves an overall accuracy of 94%, which is higher than the accuracy obtained using other CNN architectures and RF. Our method was also validated with contrast-enhanced CT scans of patients with chronic thromboembolic pulmonary hypertension to demonstrate that our model generalizes well to contrast-enhanced modalities. The proposed method outperforms state-of-the-art methods, paving the way for future use of 3-D CNN for artery/vein classification in CT images.

Quantitative 3-D morphometric analysis of individual dendritic spines

The observation and analysis of dendritic spines morphological changes poses a major challenge in neuroscience studies. The alterations of their density and/or morphology are indicators of the cellular processes involved in neural plasticity underlying learning and memory, and are symptomatic in neuropsychiatric disorders. Despite ongoing intense investigations in imaging approaches, the relationship between changes in spine morphology and synaptic function is still unknown. The existing quantitative analyses are difficult to perform and require extensive user intervention. Here, we propose a new method for (1) the three-dimensional (3-D) segmentation of dendritic spines using a multi-scale opening approach and (2) define 3-D morphological attributes of individual spines for the effective assessment of their structural plasticity. The method was validated using confocal light microscopy images of dendritic spines from dissociated hippocampal cultures and brain slices (1) to evaluate accuracy relative to manually labeled ground-truth annotations and relative to the state-of-the-art Imaris tool, (2) to analyze reproducibility of user-independence of the segmentation method, and (3) to quantitatively analyze morphological changes in individual spines before and after chemically induced long-term potentiation. The method was monitored and used to precisely describe the morphology of individual spines in real-time using consecutive images of the same dendritic fragment.

DEEP-LEARNING STRATEGY FOR PULMONARY ARTERY-VEIN CLASSIFICATION OF NON-CONTRAST CT IMAGES

Artery-vein classification on pulmonary computed tomography (CT) images is becoming of high interest in the scientific community due to the prevalence of pulmonary vascular disease that affects arteries and veins through different mechanisms. In this work, we present a novel approach to automatically segment and classify vessels from chest CT images. We use a scale-space particle segmentation to isolate vessels, and combine a convolutional neural network (CNN) to graph-cut (GC) to classify the single particles. Information about proximity of arteries to airways is learned by the network by means of a bronchus enhanced image. The methodology is evaluated on the superior and inferior lobes of the right lung of twenty clinical cases. Comparison with manual classification and a Random Forests (RF) classifier is performed. The algorithm achieves an overall accuracy of 87% when compared to manual reference, which is higher than the 73% accuracy achieved by RF.

Multiscale Opening of Conjoined Fuzzy Objects: Theory and Applications

Theoretical properties of a multi-scale opening (MSO) algorithm for two conjoined fuzzy objects are established, and its extension to separating two conjoined fuzzy objects with different intensity properties is introduced. Also, its applications to artery/vein (A/V) separation in pulmonary CT imaging and carotid vessel segmentation in CT angiograms (CTAs) of patients with intracranial aneurysms are presented. The new algorithm accounts for distinct intensity properties of individual conjoined objects by combining fuzzy distance transform (FDT), a morphologic feature, with fuzzy connectivity, a topologic feature. The algorithm iteratively opens the two conjoined objects starting at large scales and progressing toward finer scales. Results of application of the method in separating arteries and veins in a physical cast phantom of a pig lung are presented. Accuracy of the algorithm is quantitatively evaluated in terms of sensitivity and specificity on patients' CTA data sets and its performance is compared with existing methods. Reproducibility of the algorithm is examined in terms of volumetric agreement between two users' carotid vessel segmentation results. Experimental results using this algorithm on patients' CTA data demonstrate a high average accuracy of 96.3% with 95.1% sensitivity and 97.5% specificity and a high reproducibility of 94.2% average agreement between segmentation results from two mutually independent users. Approximately, twenty-five to thirty-five user-specified seeds/separators are needed for each CTA data through a custom designed graphical interface requiring an average of thirty minutes to complete carotid vascular segmentation in a patient's CTA data set.

Automated integer programming based separation of arteries and veins from thoracic CT images

Automated computer-aided analysis of lung vessels has shown to yield promising results for non-invasive diagnosis of lung diseases. To detect vascular changes which affect pulmonary arteries and veins differently, both compartments need to be identified. We present a novel, fully automatic method that separates arteries and veins in thoracic computed tomography images, by combining local as well as global properties of pulmonary vessels. We split the problem into two parts: the extraction of multiple distinct vessel subtrees, and their subsequent labeling into arteries and veins. Subtree extraction is performed with an integer program (IP), based on local vessel geometry. As naively solving this IP is time-consuming, we show how to drastically reduce computational effort by reformulating it as a Markov Random Field. Afterwards, each subtree is labeled as either arterial or venous by a second IP, using two anatomical properties of pulmonary vessels: the uniform distribution of arteries and veins, and the parallel configuration and close proximity of arteries and bronchi. We evaluate algorithm performance by comparing the results with 25 voxel-based manual reference segmentations. On this dataset, we show good performance of the subtree extraction, consisting of very few non-vascular structures (median value: 0.9%) and merged subtrees (median value: 0.6%). The resulting separation of arteries and veins achieves a median voxel-based overlap of 96.3% with the manual reference segmentations, outperforming a state-of-the-art interactive method. In conclusion, our novel approach provides an opportunity to become an integral part of computer aided pulmonary diagnosis, where artery/vein separation is important.

Pulmonary CT and MRI phenotypes that help explain chronic pulmonary obstruction disease pathophysiology and outcomes

Pulmonary x-ray computed tomographic (CT) and magnetic resonance imaging (MRI) research and development has been motivated, in part, by the quest to subphenotype common chronic lung diseases such as chronic obstructive pulmonary disease (COPD). For thoracic CT and MRI, the main COPD research tools, disease biomarkers are being validated that go beyond anatomy and structure to include pulmonary functional measurements such as regional ventilation, perfusion, and inflammation. In addition, there has also been a drive to improve spatial and contrast resolution while at the same time reducing or eliminating radiation exposure. Therefore, this review focuses on our evolving understanding of patient-relevant and clinically important COPD endpoints and how current and emerging MRI and CT tools and measurements may be exploited for their identification, quantification, and utilization. Since reviews of the imaging physics of pulmonary CT and MRI and reviews of other COPD imaging methods were previously published and well-summarized, we focus on the current clinical challenges in COPD and the potential of newly emerging MR and CT imaging measurements to address them. Here we summarize MRI and CT imaging methods and their clinical translation for generating reproducible and sensitive measurements of COPD related to pulmonary ventilation and perfusion as well as parenchyma morphology. The key clinical problems in COPD provide an important framework in which pulmonary imaging needs to rapidly move in order to address the staggering burden, costs, as well as the mortality and morbidity associated with COPD.

Automatic Synthesis of Anthropomorphic Pulmonary CT Phantoms

The great density and structural complexity of pulmonary vessels and airways impose limitations on the generation of accurate reference standards, which are critical in training and in the validation of image processing methods for features such as pulmonary vessel segmentation or artery-vein (AV) separations. The design of synthetic computed tomography (CT) images of the lung could overcome these difficulties by providing a database of pseudorealistic cases in a constrained and controlled scenario where each part of the image is differentiated unequivocally. This work demonstrates a complete framework to generate computational anthropomorphic CT phantoms of the human lung automatically. Starting from biological and image-based knowledge about the topology and relationships between structures, the system is able to generate synthetic pulmonary arteries, veins, and airways using iterative growth methods that can be merged into a final simulated lung with realistic features. A dataset of 24 labeled anthropomorphic pulmonary CT phantoms were synthesized with the proposed system. Visual examination and quantitative measurements of intensity distributions, dispersion of structures and relationships between pulmonary air and blood flow systems show good correspondence between real and synthetic lungs (p > 0.05 with low Cohen's d effect size and AUC values), supporting the potentiality of the tool and the usefulness of the generated phantoms in the biomedical image processing field.

Digital Topology and Geometry in Medical Imaging: A Survey

Digital topology and geometry refers to the use of topologic and geometric properties and features for images defined in digital grids. Such methods have been widely used in many medical imaging applications, including image segmentation, visualization, manipulation, interpolation, registration, surface-tracking, object representation, correction, quantitative morphometry etc. Digital topology and geometry play important roles in medical imaging research by enriching the scope of target outcomes and by adding strong theoretical foundations with enhanced stability, fidelity, and efficiency. This paper presents a comprehensive yet compact survey on results, principles, and insights of methods related to digital topology and geometry with strong emphasis on understanding their roles in various medical imaging applications. Specifically, this paper reviews methods related to distance analysis and path propagation, connectivity, surface-tracking, image segmentation, boundary and centerline detection, topology preservation and local topological properties, skeletonization, and object representation, correction, and quantitative morphometry. A common thread among the topics reviewed in this paper is that their theory and algorithms use the principle of digital path connectivity, path propagation, and neighborhood analysis.