Automated segmentation of pulmonary structures in thoracic computed tomography scans: a review

Computational framework for the generation of one-dimensional vascular models accounting for uncertainty in networks extracted from medical images

One-dimensional (1D) cardiovascular models offer a non-invasive method to answer medical questions, including predictions of wave-reflection, shear stress, functional flow reserve, vascular resistance and compliance. This model type can predict patient-specific outcomes by solving 1D fluid dynamics equations in geometric networks extracted from medical images. However, the inherent uncertainty in in vivo imaging introduces variability in network size and vessel dimensions, affecting haemodynamic predictions. Understanding the influence of variation in image-derived properties is essential to assess the fidelity of model predictions. Numerous programs exist to render three-dimensional surfaces and construct vessel centrelines. Still, there is no exact way to generate vascular trees from the centrelines while accounting for uncertainty in data. This study introduces an innovative framework employing statistical change point analysis to generate labelled trees that encode vessel dimensions and their associated uncertainty from medical images. To test this framework, we explore the impact of uncertainty in 1D haemodynamic predictions in a systemic and pulmonary arterial network. Simulations explore haemodynamic variations resulting from changes in vessel dimensions and segmentation; the latter is achieved by analysing multiple segmentations of the same images. Results demonstrate the importance of accurately defining vessel radii and lengths when generating high-fidelity patient-specific haemodynamics models. KEY POINTS: This study introduces novel algorithms for generating labelled directed trees from medical images, focusing on accurate junction node placement and radius extraction using change points to provide haemodynamic predictions with uncertainty within expected measurement error. Geometric features, such as vessel dimension (length and radius) and network size, significantly impact pressure and flow predictions in both pulmonary and aortic arterial networks. Standardizing networks to a consistent number of vessels is crucial for meaningful comparisons and decreases haemodynamic uncertainty. Change points are valuable to understanding structural transitions in vascular data, providing an automated and efficient way to detect shifts in vessel characteristics and ensure reliable extraction of representative vessel radii.

CAVE: Cerebral artery-vein segmentation in digital subtraction angiography

Cerebral X-ray digital subtraction angiography (DSA) is a widely used imaging technique in patients with neurovascular disease, allowing for vessel and flow visualization with high spatio-temporal resolution. Automatic artery-vein segmentation in DSA plays a fundamental role in vascular analysis with quantitative biomarker extraction, facilitating a wide range of clinical applications. The widely adopted U-Net applied on static DSA frames often struggles with disentangling vessels from subtraction artifacts. Further, it falls short in effectively separating arteries and veins as it disregards the temporal perspectives inherent in DSA. To address these limitations, we propose to simultaneously leverage spatial vasculature and temporal cerebral flow characteristics to segment arteries and veins in DSA. The proposed network, coined CAVE, encodes a 2D+time DSA series using spatial modules, aggregates all the features using temporal modules, and decodes it into 2D segmentation maps. On a large multi-center clinical dataset, CAVE achieves a vessel segmentation Dice of 0.84 (±0.04) and an artery-vein segmentation Dice of 0.79 (±0.06). CAVE surpasses traditional Frangi-based k-means clustering (P < 0.001) and U-Net (P < 0.001) by a significant margin, demonstrating the advantages of harvesting spatio-temporal features. This study represents the first investigation into automatic artery-vein segmentation in DSA using deep learning. The code is publicly available at https://github.com/RuishengSu/CAVE_DSA.

Development of an automated optimal distance feature-based decision system for diagnosing knee osteoarthritis using segmented X-ray images

Knee Osteoarthritis (KOA) is a leading cause of disability and physical inactivity. It is a degenerative joint disease that affects the cartilage, cushions the bones, and protects them from rubbing against each other during motion. If not treated early, it may lead to knee replacement. In this regard, early diagnosis of KOA is necessary for better treatment. Nevertheless, manual KOA detection is time-consuming and error-prone for large data hubs. In contrast, an automated detection system aids the specialist in diagnosing KOA grades accurately and quickly. So, the main objective of this study is to create an automated decision system that can analyze KOA and classify the severity grades, utilizing the extracted features from segmented X-ray images. In this study, two different datasets were collected from the Mendeley and Kaggle database and combined to generate a large data hub containing five classes: Grade 0 (Healthy), Grade 1 (Doubtful), Grade 2 (Minimal), Grade 3 (Moderate), and Grade 4 (Severe). Several image processing techniques were employed to segment the region of interest (ROI). These included Gradient-weighted Class Activation Mapping (Grad-Cam) to detect the ROI, cropping the ROI portion, applying histogram equalization (HE) to improve contrast, brightness, and image quality, and noise reduction (using Otsu thresholding, inverting the image, and morphological closing). Besides, the focus filtering method was utilized to eliminate unwanted images. Then, six feature sets (morphological, GLCM, statistical, texture, LBP, and proposed features) were generated from segmented ROIs. After evaluating the statistical significance of the features and selection methods, the optimal feature set (prominent six distance features) was selected, and five machine learning (ML) models were employed. Additionally, a decision-making strategy based on the six optimal features is proposed. The XGB model outperformed other models with a 99.46 % accuracy, using six distance features, and the proposed decision-making strategy was validated by testing 30 images.

A Survey on Artificial Intelligence in Pulmonary Imaging

Over the last decade, deep learning (DL) has contributed a paradigm shift in computer vision and image recognition creating widespread opportunities of using artificial intelligence in research as well as industrial applications. DL has been extensively studied in medical imaging applications, including those related to pulmonary diseases. Chronic obstructive pulmonary disease, asthma, lung cancer, pneumonia, and, more recently, COVID-19 are common lung diseases affecting nearly 7.4% of world population. Pulmonary imaging has been widely investigated toward improving our understanding of disease etiologies and early diagnosis and assessment of disease progression and clinical outcomes. DL has been broadly applied to solve various pulmonary image processing challenges including classification, recognition, registration, and segmentation. This paper presents a survey of pulmonary diseases, roles of imaging in translational and clinical pulmonary research, and applications of different DL architectures and methods in pulmonary imaging with emphasis on DL-based segmentation of major pulmonary anatomies such as lung volumes, lung lobes, pulmonary vessels, and airways as well as thoracic musculoskeletal anatomies related to pulmonary diseases.

Quantitative CT Scan Imaging of the Airways for Diagnosis and Management of Lung Disease

CT scan imaging provides high-resolution images of the lungs in patients with chronic respiratory diseases. Extensive research over the last several decades has focused on developing novel quantitative CT scan airway measurements that reflect abnormal airway structure. Despite many observational studies demonstrating that associations between CT scan airway measurements and clinically important outcomes such as morbidity, mortality, and lung function decline, few quantitative CT scan measurements are applied in clinical practice. This article provides an overview of the relevant methodologic considerations for implementing quantitative CT scan airway analyses and provides a review of the scientific literature involving quantitative CT scan airway measurements used in clinical or randomized trials and observational studies of humans. We also discuss emerging evidence for the clinical usefulness of quantitative CT scan imaging of the airways and discuss what is required to bridge the gap between research and clinical application. CT scan airway measurements continue to improve our understanding of disease pathophysiologic features, diagnosis, and outcomes. However, a literature review revealed a need for studies evaluating clinical benefit when quantitative CT scan imaging is applied in the clinical setting. Technical standards for quantitative CT scan imaging of the airways and high-quality evidence of clinical benefit from management guided by quantitative CT scan imaging of the airways are required.

Reproducibility of a combined artificial intelligence and optimal-surface graph-cut method to automate bronchial parameter extraction

OBJECTIVES

Computed tomography (CT)-based bronchial parameters correlate with disease status. Segmentation and measurement of the bronchial lumen and walls usually require significant manpower. We evaluate the reproducibility of a deep learning and optimal-surface graph-cut method to automatically segment the airway lumen and wall, and calculate bronchial parameters.

METHODS

A deep-learning airway segmentation model was newly trained on 24 Imaging in Lifelines (ImaLife) low-dose chest CT scans. This model was combined with an optimal-surface graph-cut for airway wall segmentation. These tools were used to calculate bronchial parameters in CT scans of 188 ImaLife participants with two scans an average of 3 months apart. Bronchial parameters were compared for reproducibility assessment, assuming no change between scans.

RESULTS

Of 376 CT scans, 374 (99%) were successfully measured. Segmented airway trees contained a mean of 10 generations and 250 branches. The coefficient of determination (R2) for the luminal area (LA) ranged from 0.93 at the trachea to 0.68 at the 6th generation, decreasing to 0.51 at the 8th generation. Corresponding values for Wall Area Percentage (WAP) were 0.86, 0.67, and 0.42, respectively. Bland-Altman analysis of LA and WAP per generation demonstrated mean differences close to 0; limits of agreement (LoA) were narrow for WAP and Pi10 (± 3.7% of mean) and wider for LA (± 16.4-22.8% for 2-6th generations). From the 7th generation onwards, there was a sharp decrease in reproducibility and a widening LoA.

CONCLUSION

The outlined approach for automatic bronchial parameter measurement on low-dose chest CT scans is a reliable way to assess the airway tree down to the 6th generation.

STATEMENT ON CLINICAL RELEVANCE

This reliable and fully automatic pipeline for bronchial parameter measurement on low-dose CT scans has potential applications in screening for early disease and clinical tasks such as virtual bronchoscopy or surgical planning, while also enabling the exploration of bronchial parameters in large datasets.

KEY POINTS

• Deep learning combined with optimal-surface graph-cut provides accurate airway lumen and wall segmentations on low-dose CT scans. • Analysis of repeat scans showed that the automated tools had moderate-to-good reproducibility of bronchial measurements down to the 6th generation airway. • Automated measurement of bronchial parameters enables the assessment of large datasets with less man-hours.

NCCT-CECT image synthesizers and their application to pulmonary vessel segmentation

BACKGROUND AND OBJECTIVES

Non-contrast CT (NCCT) and contrast-enhanced CT (CECT) are important diagnostic tools with distinct features and applications for chest diseases. We developed two synthesizers for the mutual synthesis of NCCT and CECT and evaluated their applications.

METHODS

Two synthesizers (S₁ and S₂) were proposed based on a generative adversarial network. S₁ generated synthetic CECT (SynCECT) from NCCT and S₂ generated synthetic NCCT (SynNCCT) from CECT. A new training procedure for synthesizers was proposed. Initially, the synthesizers were pretrained using self-supervised learning (SSL) and dual-energy CT (DECT) and then fine-tuned using the registered NCCT and CECT images. Pulmonary vessel segmentation from NCCT was used as an example to demonstrate the effectiveness of the synthesizers. Two strategies (ST₁ and ST₂) were proposed for pulmonary vessel segmentation. In ST₁, CECT images were used to train a segmentation model (Model-CECT), NCCT images were converted to SynCECT through S₁, and SynCECT was input to Model-CECT for testing. In ST₂, CECT data were converted to SynNCCT through S₂. SynNCCT and CECT-based annotations were used to train an additional model (Model-NCCT), and NCCT was input to Model-NCCT for testing. Three datasets, D1 (40 paired CTs), D2 (14 NCCTs and 14 CECTs), and D3 (49 paired DECTs), were used to evaluate the synthesizers and strategies.

RESULTS

For S₁, the mean absolute error (MAE), mean squared error (MSE), peak signal-to-noise ratio (PSNR), and structural similarity index (SSIM) were 14.60± 2.19, 1644± 890, 34.34± 1.91, and 0.94± 0.02, respectively. For S₂, they were 12.52± 2.59, 1460± 922, 35.08± 2.35, and 0.95± 0.02, respectively. Our synthesizers outperformed the counterparts of CycleGAN, Pix2Pix, and Pix2PixHD. The results of ablation studies on SSL pretraining, DECT pretraining, and fine-tuning showed that performance worsened (for example, for S₁, MAE increased to 16.53± 3.10, 17.98± 3.10, and 20.57± 3.75, respectively). Model-NCCT and Model-CECT achieved dice similarity coefficients (DSC) of 0.77 and 0.86 on D1 and 0.77 and 0.72 on D2, respectively.

CONCLUSIONS

The proposed synthesizers realized mutual and high-quality synthesis between NCCT and CECT images; the training procedures, including SSL pretraining, DECT pretraining, and fine-tuning, were critical to their effectiveness. The results demonstrated the usefulness of synthesizers for pulmonary vessel segmentation from NCCT images.

Path planning for percutaneous lung biopsy based on the loose-Pareto and adaptive heptagonal optimization method

Lung cancer has is highly prevalent worldwide and is the leading cause of cancer-related deaths. In the clinic, a biopsy sample of the lesion is taken to determine whether a lung mass is benign or malignant. CT-guided percutaneous lung biopsy is a minimally invasive intervention and is commonly used to diagnose lung cancer. Path planning before surgery plays a crucial role in percutaneous lung biopsy. Traditionally, path planning for lung biopsy is performed manually by physicians based on CT images of the patient, which demands knowledge and extensive clinical experience of the operating physicians. In this work, a computer-assisted path planning system for percutaneous lung biopsy is proposed based on clinical objectives. Five constraints are presented to remove unqualified skin entry points and determine a feasible entry region based on clinical criteria. Inspired by the Pareto principle and the concept of geometric weighting, the loose-Pareto and adaptive heptagonal optimization (LPHO) method is introduced to plan the optimal puncture path. CT images of 29 patients were collected from Zigong First People's Hospital. Retrospective experiments and test experiments were conducted to evaluate the effectiveness of the algorithm. The planning paths obtained using the proposed method were clinically feasible for 89.7% of patients, demonstrating the applicability and robustness of the system in surgical path planning for lung biopsy.

Automated Detection of Broncho-Arterial Pairs Using CT Scans Employing Different Approaches to Classify Lung Diseases

Current research indicates that for the identification of lung disorders, comprising pneumonia and COVID-19, structural distortions of bronchi and arteries (BA) should be taken into account. CT scans are an effective modality to detect lung anomalies. However, anomalies in bronchi and arteries can be difficult to detect. Therefore, in this study, alterations of bronchi and arteries are considered in the classification of lung diseases. Four approaches to highlight these are introduced: (a) a Hessian-based approach, (b) a region-growing algorithm, (c) a clustering-based approach, and (d) a color-coding-based approach. Prior to this, the lungs are segmented, employing several image preprocessing algorithms. The utilized COVID-19 Lung CT scan dataset contains three classes named Non-COVID, COVID, and community-acquired pneumonia, having 6983, 7593, and 2618 samples, respectively. To classify the CT scans into three classes, two deep learning architectures, (a) a convolutional neural network (CNN) and (b) a CNN with long short-term memory (LSTM) and an attention mechanism, are considered. Both these models are trained with the four datasets achieved from the four approaches. Results show that the CNN model achieved test accuracies of 88.52%, 87.14%, 92.36%, and 95.84% for the Hessian, the region-growing, the color-coding, and the clustering-based approaches, respectively. The CNN with LSTM and an attention mechanism model results in an increase in overall accuracy for all approaches with an 89.61%, 88.28%, 94.61%, and 97.12% test accuracy for the Hessian, region-growing, color-coding, and clustering-based approaches, respectively. To assess overfitting, the accuracy and loss curves and k-fold cross-validation technique are employed. The Hessian-based and region-growing algorithm-based approaches produced nearly equivalent outcomes. Our proposed method outperforms state-of-the-art studies, indicating that it may be worthwhile to pay more attention to BA features in lung disease classification based on CT images.

RNN-combined graph convolutional network with multi-feature fusion for tuberculosis cavity segmentation

Tuberculosis is a common infectious disease in the world. Tuberculosis cavities are common and an important imaging signs in tuberculosis. Accurate segmentation of tuberculosis cavities has practical significance for indicating the activity of lesions and guiding clinical treatment. However, this task faces challenges such as blurred boundaries, irregular shapes, different location and size of lesions and similar structures on computed tomography (CT) to other lung diseases or tissues. To overcome these problems, we propose a novel RNN-combined graph convolutional network (R2GCN) method, which integrates the bidirectional recurrent network (BRN) and graph convolution network (GCN) modules. First, feature extraction is performed on the input image by VGG-16 or ResNet-50 to obtain the feature map. The feature map is then used as the input of the two modules. On the one hand, we adopt the BRN to retrieve contextual information from the feature map. On the other hand, we take the vector for each location in the feature map as input nodes and utilize GCN to extract node topology information. Finally, two types of features obtained fuse together. Our strategy can not only make full use of node correlations and differences, but also obtain more precise segmentation boundaries. Extensive experiments on CT images of cavitary patients with tuberculosis show that our proposed method achieves the best segmentation accuracy than compared segmentation methods. Our method can be used for the diagnosis of tuberculosis cavity and the evaluation of tuberculosis cavity treatment.

A Fissure-Aided Registration Approach for Automatic Pulmonary Lobe Segmentation Using Deep Learning

The segmentation of pulmonary lobes is important in clinical assessment, lesion location, and surgical planning. Automatic lobe segmentation is challenging, mainly due to the incomplete fissures or the morphological variation resulting from lung disease. In this work, we propose a learning-based approach that incorporates information from the local fissures, the whole lung, and priori pulmonary anatomy knowledge to separate the lobes robustly and accurately. The prior pulmonary atlas is registered to the test CT images with the aid of the detected fissures. The result of the lobe segmentation is obtained by mapping the deformation function on the lobes-annotated atlas. The proposed method is evaluated in a custom dataset with COPD. Twenty-four CT scans randomly selected from the custom dataset were segmented manually and are available to the public. The experiments showed that the average dice coefficients were 0.95, 0.90, 0.97, 0.97, and 0.97, respectively, for the right upper, right middle, right lower, left upper, and left lower lobes. Moreover, the comparison of the performance with a former learning-based segmentation approach suggests that the presented method could achieve comparable segmentation accuracy and behave more robustly in cases with morphological specificity.

Assessment of Heterogeneity in Lung Structure and Function During Mechanical Ventilation: A Review of Methodologies

The mammalian lung is characterized by heterogeneity in both its structure and function, by incorporating an asymmetric branching airway tree optimized for maintenance of efficient ventilation, perfusion, and gas exchange. Despite potential benefits of naturally occurring heterogeneity in the lungs, there may also be detrimental effects arising from pathologic processes, which may result in deficiencies in gas transport and exchange. Regardless of etiology, pathologic heterogeneity results in the maldistribution of regional ventilation and perfusion, impairments in gas exchange, and increased work of breathing. In extreme situations, heterogeneity may result in respiratory failure, necessitating support with a mechanical ventilator. This review will present a summary of measurement techniques for assessing and quantifying heterogeneity in respiratory system structure and function during mechanical ventilation. These methods have been grouped according to four broad categories: (1) inverse modeling of heterogeneous mechanical function; (2) capnography and washout techniques to measure heterogeneity of gas transport; (3) measurements of heterogeneous deformation on the surface of the lung; and finally (4) imaging techniques used to observe spatially-distributed ventilation or regional deformation. Each technique varies with regard to spatial and temporal resolution, degrees of invasiveness, risks posed to patients, as well as suitability for clinical implementation. Nonetheless, each technique provides a unique perspective on the manifestations and consequences of mechanical heterogeneity in the diseased lung.

How AI Can Help in the Diagnostic Dilemma of Pulmonary Nodules

Simple Summary

Pulmonary nodules are considered a sign of bronchogenic carcinoma, detecting them early will reduce their progression and can save lives. Lung cancer is the second most common type of cancer in both men and women. This manuscript discusses the current applications of artificial intelligence (AI) in lung segmentation as well as pulmonary nodule segmentation and classification using computed tomography (CT) scans, published in the last two decades, in addition to the limitations and future prospects in the field of AI.

Abstract

Pulmonary nodules are the precursors of bronchogenic carcinoma, its early detection facilitates early treatment which save a lot of lives. Unfortunately, pulmonary nodule detection and classification are liable to subjective variations with high rate of missing small cancerous lesions which opens the way for implementation of artificial intelligence (AI) and computer aided diagnosis (CAD) systems. The field of deep learning and neural networks is expanding every day with new models designed to overcome diagnostic problems and provide more applicable and simply used models. We aim in this review to briefly discuss the current applications of AI in lung segmentation, pulmonary nodule detection and classification.

Semi-Supervised Segmentation of Radiation-Induced Pulmonary Fibrosis From Lung CT Scans With Multi-Scale Guided Dense Attention

Computed Tomography (CT) plays an important role in monitoring radiation-induced Pulmonary Fibrosis (PF), where accurate segmentation of the PF lesions is highly desired for diagnosis and treatment follow-up. However, the task is challenged by ambiguous boundary, irregular shape, various position and size of the lesions, as well as the difficulty in acquiring a large set of annotated volumetric images for training. To overcome these problems, we propose a novel convolutional neural network called PF-Net and incorporate it into a semi-supervised learning framework based on Iterative Confidence-based Refinement And Weighting of pseudo Labels (I-CRAWL). Our PF-Net combines 2D and 3D convolutions to deal with CT volumes with large inter-slice spacing, and uses multi-scale guided dense attention to segment complex PF lesions. For semi-supervised learning, our I-CRAWL employs pixel-level uncertainty-based confidence-aware refinement to improve the accuracy of pseudo labels of unannotated images, and uses image-level uncertainty for confidence-based image weighting to suppress low-quality pseudo labels in an iterative training process. Extensive experiments with CT scans of Rhesus Macaques with radiation-induced PF showed that: 1) PF-Net achieved higher segmentation accuracy than existing 2D, 3D and 2.5D neural networks, and 2) I-CRAWL outperformed state-of-the-art semi-supervised learning methods for the PF lesion segmentation task. Our method has a potential to improve the diagnosis of PF and clinical assessment of side effects of radiotherapy for lung cancers.

Texture appearance model, a new model-based segmentation paradigm, application on the segmentation of lung nodule in the CT scan of the chest

Lung cancer causes more than one million deaths worldwide each year. Averages of 5-year survival rate of patients with Non-small cell lung cancer (NSCLC), which is the most common type of lung cancer, is 15%. Computer-Aided Detection (CAD) is a very important tool for identifying lung lesions in medical imaging. In general, the process line of a CAD system can be divided into four main stages: preprocessing, localization, feature extraction, and classification. As segmentation is required for localization in computer vision and medical image analysis, this step has become a major and challenging problem, and much research has been done on new segmentation techniques. In recent years, interest in model-based segmentation methods has increased, and the reason for this is even if some object information is lost, such gaps can be filled by using the previous information in the model. This paper proposed Texture Appearance Model (TAM), which is a new model-based method and segments all types of nodule areas accurately and efficiently, including juxta-pleural nodules, without separating the lung from the surrounding area in a CT scan of the lung. In this method, Texture Representation of Image (TRI) is obtained using tissue texture feature extraction and feature selection algorithms. The proposed method has been evaluated in 85 nodules of the dataset, received from the Iranian hospital, in which the ground-truth annotation by physicians and CT imaging data were provided. The results show that the proposed algorithm has an encouraging performance for distinguishing different types of nodules, including pleural, cavity and non-solid nodules, achieving an average dice similarity coefficient (DSC) of 84.75%.

Creating a training set for artificial intelligence from initial segmentations of airways

Airways segmentation is important for research about pulmonary disease but require a large amount of time by trained specialists. We used an openly available software to improve airways segmentations obtained from an artificial intelligence (AI) tool and retrained the tool to get a better performance. Fifteen initial airway segmentations from low-dose chest computed tomography scans were obtained with a 3D-Unet AI tool previously trained on Danish Lung Cancer Screening Trial and Erasmus-MC Sophia datasets. Segmentations were manually corrected in 3D Slicer. The corrected airway segmentations were used to retrain the 3D-Unet. Airway measurements were automatically obtained and included count, airway length and luminal diameter per generation from the segmentations. Correcting segmentations required 2–4 h per scan. Manually corrected segmentations had more branches (p < 0.001), longer airways (p < 0.001) and smaller luminal diameters (p = 0.004) than initial segmentations. Segmentations from retrained 3D-Unets trended towards more branches and longer airways compared to the initial segmentations. The largest changes were seen in airways from 6th generation onwards. Manual correction results in significantly improved segmentations and is potentially a useful and time-efficient method to improve the AI tool performance on a specific hospital or research dataset.

Automatic airway segmentation from computed tomography using robust and efficient 3-D convolutional neural networks

This paper presents a fully automatic and end-to-end optimised airway segmentation method for thoracic computed tomography, based on the U-Net architecture. We use a simple and low-memory 3D U-Net as backbone, which allows the method to process large 3D image patches, often comprising full lungs, in a single pass through the network. This makes the method simple, robust and efficient. We validated the proposed method on three datasets with very different characteristics and various airway abnormalities: (1) a dataset of pediatric patients including subjects with cystic fibrosis, (2) a subset of the Danish Lung Cancer Screening Trial, including subjects with chronic obstructive pulmonary disease, and (3) the EXACT'09 public dataset. We compared our method with other state-of-the-art airway segmentation methods, including relevant learning-based methods in the literature evaluated on the EXACT'09 data. We show that our method can extract highly complete airway trees with few false positive errors, on scans from both healthy and diseased subjects, and also that the method generalizes well across different datasets. On the EXACT'09 test set, our method achieved the second highest sensitivity score among all methods that reported good specificity.

Backward Planning for a Multi-Stage Steerable Needle Lung Robot

Lung cancer is one of the deadliest types of cancer, and early diagnosis is crucial for successful treatment. Definitively diagnosing lung cancer typically requires biopsy, but current approaches either carry a high procedural risk for the patient or are incapable of reaching many sites of clinical interest in the lung. We present a new sampling-based planning method for a steerable needle lung robot that has the potential to accurately reach targets in most regions of the lung. The robot comprises three stages: a transorally deployed bronchoscope, a sharpened piercing tube (to pierce into the lung parenchyma from the airways), and a steerable needle able to navigate to the target. Planning for the sequential deployment of all three stages under health safety concerns is a challenging task, as each stage depends on the previous one. We introduce a new backward planning approach that starts at the target and advances backwards toward the airways with the goal of finding a piercing site reachable by the bronchoscope. This new strategy enables faster performance by iteratively building a single search tree during the entire computation period, whereas previous forward approaches have relied on repeating this expensive tree construction process many times. Additionally, our method further reduces runtime by employing biased sampling and sample rejection based on geometric constraints. We evaluate this approach using simulation-based studies in anatomical lung models. We demonstrate in comparison with existing techniques that the new approach (i) is more likely to find a path to a target, (ii) is more efficient by reaching targets more than 5 times faster on average, and (iii) arrives at lower-risk paths in shorter time.

Pathological lung segmentation in chest CT images based on improved random walker

BACKGROUND AND OBJECTIVE

Pathological lung segmentation as a pretreatment step in the diagnosis of lung diseases has been widely explored. Because of the complexity of pathological lung structures and gray blur of the border, accurate lung segmentation in clinical 3D computed tomography images is a challenging task. In view of the current situation, the work proposes a fast and accurate pathological lung segmentation method. The following contributions have been made: First, the edge weights introduce spatial information and clustering information, so that walkers can use more image information during walking. Second, a Gaussian Distribution of seed point set is established to further expand the possibility of selection between fake seed points and real seed points. Finally, the pre-parameter is calculated using original seed points, and the final results are fitted with new seed points.

METHODS

This study proposes a segmentation method based on an improved random walker algorithm. The proposed method consists of the following steps: First, a gray value is used as the sample distribution. Gaussian mixture model is used to obtain the clustering probability of an image. Thus, the spatial distance and clustering result are added as new weights, and the new edge weights are used to construct a random walker map. Second, a large number of marked points are automatically selected, and the intermediate results are obtained from the newly constructed map and retained only as pre-parameters. When new seed points are introduced, the probability value of the walker is quickly calculated from the new parameters and pre-parameters, and the final segmentation result can be obtained.

RESULTS

The proposed method was tested on 65 sets of CT cases. Quantitative evaluation with different methods confirms the high accuracy on our dataset (98.55%) and LOLA11 dataset (97.41%). Similarly, the average segmentation time (10.5s) is faster than random walker (1,332.5s).

CONCLUSIONS

The comparison of the experimental results show that the proposed method can accurately and quickly obtain pathological lung processing results. Therefore, it has potential clinical applications.

A CT-Based Automated Algorithm for Airway Segmentation Using Freeze-and-Grow Propagation and Deep Learning

Chronic obstructive pulmonary disease (COPD) is a common lung disease, and quantitative CT-based bronchial phenotypes are of increasing interest as a means of exploring COPD sub-phenotypes, establishing disease progression, and evaluating intervention outcomes. Reliable, fully automated, and accurate segmentation of pulmonary airway trees is critical to such exploration. We present a novel approach of multi-parametric freeze-and-grow (FG) propagation which starts with a conservative segmentation parameter and captures finer details through iterative parameter relaxation. First, a CT intensity-based FG algorithm is developed and applied for airway tree segmentation. A more efficient version is produced using deep learning methods generating airway lumen likelihood maps from CT images, which are input to the FG algorithm. Both CT intensity- and deep learning-based algorithms are fully automated, and their performance, in terms of repeat scan reproducibility, accuracy, and leakages, is evaluated and compared with results from several state-of-the-art methods including an industry-standard one, where segmentation results were manually reviewed and corrected. Both new algorithms show a reproducibility of 95% or higher for total lung capacity (TLC) repeat CT scans. Experiments on TLC CT scans from different imaging sites at standard and low radiation dosages show that both new algorithms outperform the other methods in terms of leakages and branch-level accuracy. Considering the performance and execution times, the deep learning-based FG algorithm is a fully automated option for large multi-site studies.

Automated vessel segmentation in lung CT and CTA images via deep neural networks

BACKGROUND

The distribution of pulmonary vessels in computed tomography (CT) and computed tomography angiography (CTA) images of lung is important for diagnosing disease, formulating surgical plans and pulmonary research.

PURPOSE

Based on the pulmonary vascular segmentation task of International Symposium on Image Computing and Digital Medicine 2020 challenge, this paper reviews 12 different pulmonary vascular segmentation algorithms of lung CT and CTA images and then objectively evaluates and compares their performances.

METHODS

First, we present the annotated reference dataset of lung CT and CTA images. A subset of the dataset consisting 7,307 slices for training and 3,888 slices for testing was made available for participants. Second, by analyzing the performance comparison of different convolutional neural networks from 12 different institutions for pulmonary vascular segmentation, the reasons for some defects and improvements are summarized. The models are mainly based on U-Net, Attention, GAN, and multi-scale fusion network. The performance is measured in terms of Dice coefficient, over segmentation rate and under segmentation rate. Finally, we discuss several proposed methods to improve the pulmonary vessel segmentation results using deep neural networks.

RESULTS

By comparing with the annotated ground truth from both lung CT and CTA images, most of 12 deep neural network algorithms do an admirable job in pulmonary vascular extraction and segmentation with the dice coefficients ranging from 0.70 to 0.85. The dice coefficients for the top three algorithms are about 0.80.

CONCLUSIONS

Study results show that integrating methods that consider spatial information, fuse multi-scale feature map, or have an excellent post-processing to deep neural network training and optimization process are significant for further improving the accuracy of pulmonary vascular segmentation.

Internet of Medical Things: An Effective and Fully Automatic IoT Approach Using Deep Learning and Fine-Tuning to Lung CT Segmentation

Several pathologies have a direct impact on society, causing public health problems. Pulmonary diseases such as Chronic obstructive pulmonary disease (COPD) are already the third leading cause of death in the world, leaving tuberculosis at ninth with 1.7 million deaths and over 10.4 million new occurrences. The detection of lung regions in images is a classic medical challenge. Studies show that computational methods contribute significantly to the medical diagnosis of lung pathologies by Computerized Tomography (CT), as well as through Internet of Things (IoT) methods based in the context on the health of things. The present work proposes a new model based on IoT for classification and segmentation of pulmonary CT images, applying the transfer learning technique in deep learning methods combined with Parzen’s probability density. The proposed model uses an Application Programming Interface (API) based on the Internet of Medical Things to classify lung images. The approach was very effective, with results above 98% accuracy for classification in pulmonary images. Then the model proceeds to the lung segmentation stage using the Mask R-CNN network to create a pulmonary map and use fine-tuning to find the pulmonary borders on the CT image. The experiment was a success, the proposed method performed better than other works in the literature, reaching high segmentation metrics values such as accuracy of 98.34%. Besides reaching 5.43 s in segmentation time and overcoming other transfer learning models, our methodology stands out among the others because it is fully automatic. The proposed approach has simplified the segmentation process using transfer learning. It has introduced a faster and more effective method for better-performing lung segmentation, making our model fully automatic and robust.

DCT-MIL: Deep CNN transferred multiple instance learning for COPD identification using CT images

While many pre-defined computed tomographic (CT) measures have been utilized to characterize chronic obstructive pulmonary disease (COPD), it is still challenging to represent pathological alternations of multiple dimensions and highly spatial heterogeneity. Deep CNN transferred multiple instance learning (DCT-MIL) is proposed to identify COPD via CT images. After the lung is divided into eight sections along the axial direction, one random axial CT image is taken out from each section as one instance. With one instance as the input, the activations of neural layers of AlexNet trained by natural images are extracted as features. After dimension reduction through principle component analysis, features of all instances are input into three MIL methods: Citation k-Nearest-Neighbor (Citation-KNN), multiple instance support vector machine, and expectation-maximization diverse density. Moreover, the performance dependence of the resulted models on the depth of the neural layer where activations are extracted and the number of features is investigated. The proposed DCT-MIL achieves an exceptional performance with an accuracy of 99.29% and area under curve of 0.9826 while using 100 principle components of features extracted from the fourth convolutional layer and Citation-KNN. It outperforms not only DCT-MIL models using other settings and the pre-trained AlexNet with fine-tuning by montages of eight lung CT images, but also other state-of-art methods. Deep CNN transferred multiple instance learning is suited for identification of COPD using CT images. It can help finding subgroups with high risk of COPD from large populations through CT scans ordered doing lung cancer screening.

MDL-IWS: Multi-view Deep Learning with Iterative Watershed for Pulmonary Fissure Segmentation

Pulmonary fissure segmentation is important for localization of lung lesions which include nodules at respective lobar territories. This can be very useful for diagnosis as well as treatment planning. In this paper, we propose a novel coarse-to-fine fissure segmentation approach by proposing a Multi-View Deep Learning driven Iterative WaterShed Algorithm (MDL-IWS). Coarse fissure segmentation obtained from multi-view deep learning yields incomplete fissure volume of interest (VOI) with additional false positives. An iterative watershed algorithm (IWS) is presented to achieve fine segmentation of fissure surfaces. As a part of the IWS algorithm, surface fitting is used to generate a more accurate fissure VOI with substantial reduction in false positives. Additionally, a weight map is used to reduce the over-segmentation of watershed in subsequent iterations. Experiments on the publicly available LOLA11 dataset clearly reveal that our method outperforms several state-of-the-art competitors.

On Modelling Label Uncertainty in Deep Neural Networks: Automatic Estimation of Intra- Observer Variability in 2D Echocardiography Quality Assessment

Uncertainty of labels in clinical data resulting from intra-observer variability can have direct impact on the reliability of assessments made by deep neural networks. In this paper, we propose a method for modelling such uncertainty in the context of 2D echocardiography (echo), which is a routine procedure for detecting cardiovascular disease at point-of-care. Echo imaging quality and acquisition time is highly dependent on the operator's experience level. Recent developments have shown the possibility of automating echo image quality quantification by mapping an expert's assessment of quality to the echo image via deep learning techniques. Nevertheless, the observer variability in the expert's assessment can impact the quality quantification accuracy. Here, we aim to model the intra-observer variability in echo quality assessment as an aleatoric uncertainty modelling regression problem with the introduction of a novel method that handles the regression problem with categorical labels. A key feature of our design is that only a single forward pass is sufficient to estimate the level of uncertainty for the network output. Compared to the 0.11 ± 0.09 absolute error (in a scale from 0 to 1) archived by the conventional regression method, the proposed method brings the error down to 0.09 ± 0.08, where the improvement is statistically significant and equivalents to 5.7% test accuracy improvement. The simplicity of the proposed approach means that it could be generalized to other applications of deep learning in medical imaging, where there is often uncertainty in clinical labels.

Clinical Implementation of Deep Learning in Thoracic Radiology: Potential Applications and Challenges

Chest X-ray radiography and computed tomography, the two mainstay modalities in thoracic radiology, are under active investigation with deep learning technology, which has shown promising performance in various tasks, including detection, classification, segmentation, and image synthesis, outperforming conventional methods and suggesting its potential for clinical implementation. However, the implementation of deep learning in daily clinical practice is in its infancy and facing several challenges, such as its limited ability to explain the output results, uncertain benefits regarding patient outcomes, and incomplete integration in daily workflow. In this review article, we will introduce the potential clinical applications of deep learning technology in thoracic radiology and discuss several challenges for its implementation in daily clinical practice.

Simulation of bronchial airway acoustics in healthy and asthmatic subjects

The onset and development of many airway pathologies affect sound propagation throughout the respiratory system; changes in respiratory sounds are detected primarily by auscultation, which is highly skill dependent. The aim of the present study was to compare healthy and asthmatic pulmonary acoustics by applying a 1D model of wave propagation on CT-based patient-specific geometries. High-resolution CT lung images were acquired in five healthy volunteers and five asthmatic patients at total lung capacity (TLC) and functional residual capacity (FRC). Tracheobronchial trees were reconstructed from CT images. Acoustic pressure, impedance and wall radial velocity were measured by simulating acoustic wave propagation of two external, acoustic pressure waves (1 Pa, 200 and 600 Hz) from the trachea level to the 4th generation. In asthmatic patients, acoustic pressure averaged across the last three generations showed a reduction equal to 29.7% (p<0.01) at FRC, at 200 Hz; input and terminal impedance were 34.5% (p<0.05) higher both at FRC and TLC; wall radial velocity was more than 80% (p<0.05) lower in higher generations both at FRC and TLC. Airway differences in asthma alter acoustic parameters at FRC and TLC, with the greatest difference at FRC and 200 Hz. Acoustic wave propagation analysis represents a quantitative approach that has potential to objectively characterize airway differences in individuals with diseases such as asthma.

A predictive factor for patients with acute respiratory distress syndrome: CT lung volumetry of the well-aerated region as an automated method

PURPOSE

Acute respiratory distress syndrome (ARDS) is an acute inflammatory lung injury that frequently shows fatal outcomes. As radiographic predictive factors, some reports have focused on the region of ill-aerated lung, but none have focused on well-aerated lung. Our objective was to evaluate the relationship between computed tomography (CT) volume of the well-aerated lung region and prognosis in patients with ARDS.

METHOD

This retrospective observational study of a single intensive care unit (ICU) included patients with ARDS treated between April 2011 and May 2013. We identified 42 patients with ARDS for whom adequate helical CT scans were available. CT images were analyzed for 3-dimensional reconstruction, and lung region volumes were measured using automated volumetry methods. Lung regions were identified by CT attenuation in Hounsfield units (HU).

RESULTS

Of the 42 patients, 35 (83.3 %) survived 28 days and 32 (76.2 %) survived to ICU discharge. CT lung volumetry was performed within 144.5 ± 76.6 s, and inter-rater reliability of CT lung volumetry for lung regions below -500 HU (well-aerated lung region) were near-perfect. Well-aerated lung region showed a positive correlation with 28-day survival (P = 0.020), and lung volumes below -900 HU correlated positively with 28-day survival and ICU survival, respectively (P = 0.028, 0.017). Survival outcome was better for percentage of well-aerated lung region/predicted total lung capacity ≥40 % than for <40 % (P = 0.039).

CONCLUSIONS

CT lung volumetry of the well-aerated lung region using an automated method allows fast, reliable quantitative CT analysis and potentially prediction of the clinical course in patients with ARDS.

Influence of image segmentation on one-dimensional fluid dynamics predictions in the mouse pulmonary arteries

Computational fluid dynamics (CFD) models are emerging tools for assisting in diagnostic assessment of cardiovascular disease. Recent advances in image segmentation have made subject-specific modelling of the cardiovascular system a feasible task, which is particularly important in the case of pulmonary hypertension, requiring a combination of invasive and non-invasive procedures for diagnosis. Uncertainty in image segmentation propagates to CFD model predictions, making the quantification of segmentation-induced uncertainty crucial for subject-specific models. This study quantifies the variability of one-dimensional CFD predictions by propagating the uncertainty of network geometry and connectivity to blood pressure and flow predictions. We analyse multiple segmentations of a single, excised mouse lung using different pre-segmentation parameters. A custom algorithm extracts vessel length, vessel radii and network connectivity for each segmented pulmonary network. Probability density functions are computed for vessel radius and length and then sampled to propagate uncertainties to haemodynamic predictions in a fixed network. In addition, we compute the uncertainty of model predictions to changes in network size and connectivity. Results show that variation in network connectivity is a larger contributor to haemodynamic uncertainty than vessel radius and length.

Automatic quantitative analysis of pulmonary vascular morphology in CT images

PURPOSE

Vascular remodeling is a significant pathological feature of various pulmonary diseases, which may be assessed by quantitative computed tomography (CT) imaging. The purpose of this study was therefore to develop and validate an automatic method for quantifying pulmonary vascular morphology in CT images.

METHODS

The proposed method consists of pulmonary vessel extraction and quantification. For extracting pulmonary vessels, a graph-cuts-based method is proposed which considers appearance (CT intensity) and shape (vesselness from a Hessian-based filter) features, and incorporates distance to the airways into the cost function to prevent false detection of airway walls. For quantifying the extracted pulmonary vessels, a radius histogram is generated by counting the occurrence of vessel radii, calculated from a distance transform-based method. Subsequently, two biomarkers, slope α and intercept β, are calculated by linear regression on the radius histogram. A public data set from the VESSEL12 challenge was used to independently evaluate the vessel extraction. The quantitative analysis method was validated using images of a three-dimensional (3D) printed vessel phantom, scanned by a clinical CT scanner and a micro-CT scanner (to obtain a gold standard). To confirm the association between imaging biomarkers and pulmonary function, 77 scleroderma patients were investigated with the proposed method.

RESULTS

In the independent evaluation with the public data set, our vessel segmentation method obtained an area under the receiver operating characteristic (ROC) curve of 0.976. The median radius difference between clinical and micro-CT scans of a 3D printed vessel phantom was 0.062 ± 0.020 mm, with interquartile range of 0.199 ± 0.050 mm. In the studied patient group, a significant correlation between diffusion capacity for carbon monoxide and the biomarkers, α (R = -0.27, P = 0.018) and β (R = 0.321, P = 0.004), was obtained.

CONCLUSION

In conclusion, the proposed method was validated independently using a public data set resulting in an area under the ROC curve of 0.976 and using a 3D printed vessel phantom data set, showing a vessel sizing error of 0.062 mm (0.16 in-plane pixel units). The correlation between imaging biomarkers and diffusion capacity in a clinical data set confirmed an association between lung structure and function. This quantification of pulmonary vascular morphology may be helpful in understanding the pathophysiology of pulmonary vascular diseases.

Imaging the Injured Lung: Mechanisms of Action and Clinical Use

Acute respiratory distress syndrome (ARDS) consists of acute hypoxemic respiratory failure characterized by massive and heterogeneously distributed loss of lung aeration caused by diffuse inflammation and edema present in interstitial and alveolar spaces. It is defined by consensus criteria, which include diffuse infiltrates on chest imaging-either plain radiography or computed tomography. This review will summarize how imaging sciences can inform modern respiratory management of ARDS and continue to increase the understanding of the acutely injured lung. This review also describes newer imaging methodologies that are likely to inform future clinical decision-making and potentially improve outcome. For each imaging modality, this review systematically describes the underlying principles, technology involved, measurements obtained, insights gained by the technique, emerging approaches, limitations, and future developments. Finally, integrated approaches are considered whereby multimodal imaging may impact management of ARDS.

Automatic Pathological Lung Segmentation in Low-Dose CT Image Using Eigenspace Sparse Shape Composition

The segmentation of lungs with severe pathology is a nontrivial problem in the clinical application. Due to complex structures, pathological changes, individual differences, and low image quality, accurate lung segmentation in clinical 3-D computed tomography (CT) images is still a challenging task. To overcome these problems, a novel dictionary-based approach is introduced to automatically segment pathological lungs in 3-D low-dose CT images. Sparse shape composition is integrated with the eigenvector space shape prior model, called eigenspace sparse shape composition, to reduce local shape reconstruction error caused by the weak and misleading appearance prior information. To initialize the shape model, a landmark recognition method based on discriminative appearance dictionary is introduced to handle lesions and local details. Furthermore, a new vertex search strategy based on the gradient vector flow field is also proposed to drive the shape deformation to the target boundary. The proposed algorithm is tested on 78 3-D low-dose CT images with lung tumors. Compared to the state-of-the-art methods, the proposed approach can robustly and accurately detect pathological lung surface.

Segmentation of lung parenchyma in CT images using CNN trained with the clustering algorithm generated dataset

Background

Lung segmentation constitutes a critical procedure for any clinical-decision supporting system aimed to improve the early diagnosis and treatment of lung diseases. Abnormal lungs mainly include lung parenchyma with commonalities on CT images across subjects, diseases and CT scanners, and lung lesions presenting various appearances. Segmentation of lung parenchyma can help locate and analyze the neighboring lesions, but is not well studied in the framework of machine learning.

Methods

We proposed to segment lung parenchyma using a convolutional neural network (CNN) model. To reduce the workload of manually preparing the dataset for training the CNN, one clustering algorithm based method is proposed firstly. Specifically, after splitting CT slices into image patches, the k-means clustering algorithm with two categories is performed twice using the mean and minimum intensity of image patch, respectively. A cross-shaped verification, a volume intersection, a connected component analysis and a patch expansion are followed to generate final dataset. Secondly, we design a CNN architecture consisting of only one convolutional layer with six kernels, followed by one maximum pooling layer and two fully connected layers. Using the generated dataset, a variety of CNN models are trained and optimized, and their performances are evaluated by eightfold cross-validation. A separate validation experiment is further conducted using a dataset of 201 subjects (4.62 billion patches) with lung cancer or chronic obstructive pulmonary disease, scanned by CT or PET/CT. The segmentation results by our method are compared with those yielded by manual segmentation and some available methods.

Results

A total of 121,728 patches are generated to train and validate the CNN models. After the parameter optimization, our CNN model achieves an average F-score of 0.9917 and an area of curve up to 0.9991 for classification of lung parenchyma and non-lung-parenchyma. The obtain model can segment the lung parenchyma accurately for 201 subjects with heterogeneous lung diseases and CT scanners. The overlap ratio between the manual segmentation and the one by our method reaches 0.96.

Conclusions

The results demonstrated that the proposed clustering algorithm based method can generate the training dataset for CNN models. The obtained CNN model can segment lung parenchyma with very satisfactory performance and have the potential to locate and analyze lung lesions.

Technical challenges of quantitative chest MRI data analysis in a large cohort pediatric study

Objectives

This study was conducted in order to evaluate the effect of geometric distortion (GD) on MRI lung volume quantification and evaluate available manual, semi-automated, and fully automated methods for lung segmentation.

Methods

A phantom was scanned with MRI and CT. GD was quantified as the difference in phantom’s volume between MRI and CT, with CT as gold standard. Dice scores were used to measure overlap in shapes. Furthermore, 11 subjects from a prospective population-based cohort study each underwent four chest MRI acquisitions. The resulting 44 MRI scans with 2D and 3D Gradwarp were used to test five segmentation methods. Intraclass correlation coefficient, Bland–Altman plots, Wilcoxon, Mann–Whitney U, and paired t tests were used for statistics.

Results

Using phantoms, volume differences between CT and MRI varied according to MRI positions and 2D and 3D Gradwarp correction. With the phantom located at the isocenter, MRI overestimated the volume relative to CT by 5.56 ± 1.16 to 6.99 ± 0.22% with body and torso coils, respectively. Higher Dice scores and smaller intraobject differences were found for 3D Gradwarp MR images. In subjects, semi-automated and fully automated segmentation tools showed high agreement with manual segmentations (ICC = 0.971–0.993 for end-inspiratory scans; ICC = 0.992–0.995 for end-expiratory scans). Manual segmentation time per scan was approximately 3–4 h and 2–3 min for fully automated methods.

Conclusions

Volume overestimation of MRI due to GD can be quantified. Semi-automated and fully automated segmentation methods allow accurate, reproducible, and fast lung volume quantification. Chest MRI can be a valid radiation-free imaging modality for lung segmentation and volume quantification in large cohort studies.

Key Points

• Geometric distortion varies according to MRI setting and patient positioning.

• Automated segmentation methods allow fast and accurate lung volume quantification.

• MRI is a valid radiation-free alternative to CT for quantitative data analysis.

Electronic supplementary material

The online version of this article (10.1007/s00330-018-5863-7) contains supplementary material, which is available to authorized users.

A graph-cut approach for pulmonary artery-vein segmentation in noncontrast CT images

Lung vessel segmentation has been widely explored by the biomedical image processing community; however, the differentiation of arterial from venous irrigation is still a challenge. Pulmonary artery-vein (AV) segmentation using computed tomography (CT) is growing in importance owing to its undeniable utility in multiple cardiopulmonary pathological states, especially those implying vascular remodelling, allowing the study of both flow systems separately. We present a new framework to approach the separation of tree-like structures using local information and a specifically designed graph-cut methodology that ensures connectivity as well as the spatial and directional consistency of the derived subtrees. This framework has been applied to the pulmonary AV classification using a random forest (RF) pre-classifier to exploit the local anatomical differences of arteries and veins. The evaluation of the system was performed using 192 bronchopulmonary segment phantoms, 48 anthropomorphic pulmonary CT phantoms, and 26 lungs from noncontrast CT images with precise voxel-based reference standards obtained by manually labelling the vessel trees. The experiments reveal a relevant improvement in the accuracy ( ∼ 20%) of the vessel particle classification with the proposed framework with respect to using only the pre-classification based on local information applied to the whole area of the lung under study. The results demonstrated the accurate differentiation between arteries and veins in both clinical and synthetic cases, specifically when the image quality can guarantee a good airway segmentation, which opens a huge range of possibilities in the clinical study of cardiopulmonary diseases.

Improvement of fully automated airway segmentation on volumetric computed tomographic images using a 2.5 dimensional convolutional neural net

We propose a novel airway segmentation method in volumetric chest computed tomography (CT) and evaluate its performance on multiple datasets. The segmentation is performed voxel-by-voxel by a 2.5D convolutional neural net (2.5D CNN) trained in a supervised manner. To enhance the accuracy of the segmented airway tree, we simultaneously took three adjacent slices in each of the orthogonal directions including axial, sagittal, and coronal and fine-tuned the parameters that influence the tree length and the number of leakage. The gold standard of airway segmentation was generated by a semi-automated method using AVIEW™. The 2.5D CNN was trained and evaluated on a subset of inspiratory thoracic CT scans taken from the Korean obstructive lung disease study, which includes normal subjects and chronic obstructive pulmonary disease patients. The reliability and further practicality of our proposed method was demonstrated in multiple datasets. In eight test datasets collected by the same imaging protocol, the percentage detected tree length, false positive rate, and Dice similarity coefficient of our method were 92.16%, 7.74%, and 0.8997 ± 0.0892, respectively. In 20 test datasets of the EXACT'09 challenge, the percentage detected tree length was 60.1% and the false positive rate was 4.56%. Our fully automated (end-to-end) segmentation method could be applied in radiologic practice.

Safe Motion Planning for Steerable Needles Using Cost Maps Automatically Extracted from Pulmonary Images

Lung cancer is the deadliest form of cancer, and early diagnosis is critical to favorable survival rates. Definitive diagnosis of lung cancer typically requires needle biopsy. Common lung nodule biopsy approaches either carry significant risk or are incapable of accessing large regions of the lung, such as in the periphery. Deploying a steerable needle from a bronchoscope and steering through the lung allows for safe biopsy while improving the accessibility of lung nodules in the lung periphery. In this work, we present a method for extracting a cost map automatically from pulmonary CT images, and utilizing the cost map to efficiently plan safe motions for a steerable needle through the lung. The cost map encodes obstacles that should be avoided, such as the lung pleura, bronchial tubes, and large blood vessels, and additionally formulates a cost for the rest of the lung which corresponds to an approximate likelihood that a blood vessel exists at each location in the anatomy. We then present a motion planning approach that utilizes the cost map to generate paths that minimize accumulated cost while safely reaching a goal location in the lung.

The potential for machine learning algorithms to improve and reduce the cost of 3-dimensional printing for surgical planning

INTRODUCTION

3D-printed anatomical models play an important role in medical and research settings. The recent successes of 3D anatomical models in healthcare have led many institutions to adopt the technology. However, there remain several issues that must be addressed before it can become more wide-spread. Of importance are the problems of cost and time of manufacturing. Machine learning (ML) could be utilized to solve these issues by streamlining the 3D modeling process through rapid medical image segmentation and improved patient selection and image acquisition. The current challenges, potential solutions, and future directions for ML and 3D anatomical modeling in healthcare are discussed.

AREAS COVERED

This review covers research articles in the field of machine learning as related to 3D anatomical modeling. Topics discussed include automated image segmentation, cost reduction, and related time constraints.

EXPERT COMMENTARY

ML-based segmentation of medical images could potentially improve the process of 3D anatomical modeling. However, until more research is done to validate these technologies in clinical practice, their impact on patient outcomes will remain unknown. We have the necessary computational tools to tackle the problems discussed. The difficulty now lies in our ability to collect sufficient data.

Small airway segmentation in thoracic computed tomography scans: a machine learning approach

Small airway obstruction is a main cause for chronic obstructive pulmonary disease (COPD). We propose a novel method based on machine learning to extract the airway system from a thoracic computed tomography (CT) scan. The emphasis of the proposed method is on including the smallest airways that are still visible on CT. We used an optimized sampling procedure to extract airway and non-airway voxel samples from a large set of scans for which a semi-automatically constructed reference standard was available. We created a set of features which represent tubular and texture properties that are characteristic for small airway voxels. A random forest classifier was used to determine for each voxel if it belongs to the airway class. Our method was validated on a set of 20 clinical thoracic CT scans from the COPDGene study. Experiments show that our method is effective in extracting the full airway system and in detecting a large number of small airways that were missed by the semi-automatically constructed reference standard.

Healthy Lung Vessel Morphology Derived From Thoracic Computed Tomography

Knowledge of the lung vessel morphology in healthy subjects is necessary to improve our understanding about the functional network of the lung and to recognize pathologic deviations beyond the normal inter-subject variation. Established values of normal lung morphology have been derived from necropsy material of only very few subjects. In order to determine morphologic readouts from a large number of healthy subjects, computed tomography pulmonary angiography (CTPA) datasets, negative for pulmonary embolism, and other thoracic pathologies, were analyzed using a fully-automatic, in-house developed artery/vein separation algorithm. The number, volume, and tortuosity of the vessels in a diameter range between 2 and 10 mm were determined. Visual inspection of all datasets was used to exclude subjects with poor image quality or inadequate artery/vein separation from the analysis. Validation of the algorithm was performed manually by a radiologist on randomly selected subjects. In 123 subjects (men/women: 55/68), aged 59 ± 17 years, the median overlap between visual inspection and fully-automatic segmentation was 94.6% (69.2–99.9%). The median number of vessel segments in the ranges of 8–10, 6–8, 4–6, and 2–4 mm diameter was 9, 34, 134, and 797, respectively. Number of vessel segments divided by the subject's lung volume was 206 vessels/L with arteries and veins contributing almost equally. In women this vessel density was about 15% higher than in men. Median arterial and venous volumes were 1.52 and 1.54% of the lung volume, respectively. Tortuosity was best described with the sum-of-angles metric and was 142.1 rad/m (138.3–144.5 rad/m). In conclusion, our fully-automatic artery/vein separation algorithm provided reliable measures of pulmonary arteries and veins with respect to age and gender. There was a large variation between subjects in all readouts. No relevant dependence on age, gender, or vessel type was observed. These data may provide reference values for morphometric analysis of lung vessels.

Modeling Lung Architecture in the XCAT Series of Phantoms: Physiologically Based Airways, Arteries and Veins

The purpose of this paper was to extend the extended cardiac-torso (XCAT) series of computational phantoms to include a detailed lung architecture including airways and pulmonary vasculature. Eleven XCAT phantoms of varying anatomy were used in this paper. The lung lobes and initial branches of the airways, pulmonary arteries, and veins were previously defined in each XCAT model. These models were extended from the initial branches of the airways and vessels to the level of terminal branches using an anatomically-based volume-filling branching algorithm. This algorithm grew the airway and vasculature branches separately and iteratively without intersecting each other using cylindrical models with diameters estimated by order-based anatomical measurements. Geometrical features of the extended branches were compared with the literature anatomy values to quantitatively evaluate the models. These features include branching angle, length to diameter ratio, daughter to parent diameter ratio, asymmetrical branching pattern, diameter, and length ratios. The XCAT phantoms were then used to simulate CT images to qualitatively compare them with the original phantom images. The proposed growth model produced 46369 ± 12521 airways, 44737 ± 11773 arteries, and 39819 ± 9988 veins to the XCAT phantoms. Furthermore, the growth model was shown to produce asymmetrical airway, artery, and vein networks with geometrical attributes close to morphometry and model based studies. The simulated CT images of the phantoms were judged to be more realistic, including more airways and pulmonary vessels compared with the original phantoms. Future work will seek to add a heterogeneous parenchymal background into the XCAT lungs to make the phantoms even more representative of human anatomy, paving the way towards the use of XCAT models as a tool to virtually evaluate the current and emerging medical imaging technologies.

Validation of the Gatortail method for accurate sizing of pulmonary vessels from 3D medical images

PURPOSE

Detailed characterization of changes in vessel size is crucial for the diagnosis and management of a variety of vascular diseases. Because clinical measurement of vessel size is typically dependent on the radiologist's subjective interpretation of the vessel borders, it is often prone to high inter- and intra-user variability. Automatic methods of vessel sizing have been developed for two-dimensional images but a fully three-dimensional (3D) method suitable for vessel sizing from volumetric X-ray computed tomography (CT) or magnetic resonance imaging has heretofore not been demonstrated and validated robustly.

METHODS

In this paper, we refined and objectively validated Gatortail, a method that creates a mathematical geometric 3D model of each branch in a vascular tree, simulates the appearance of the virtual vascular tree in a 3D CT image, and uses the similarity of the simulated image to a patient's CT scan to drive the optimization of the model parameters, including vessel size, to match that of the patient. The method was validated with a 2-dimensional virtual tree structure under deformation, and with a realistic 3D-printed vascular phantom in which the diameter of 64 branches were manually measured 3 times each. The phantom was then scanned on a conventional clinical CT imaging system and the images processed with the in-house software to automatically segment and mathematically model the vascular tree, label each branch, and perform the Gatortail optimization of branch size and trajectory. Previously proposed methods of vessel sizing using matched Gaussian filters and tubularity metrics were also tested. The Gatortail method was then demonstrated on the pulmonary arterial tree segmented from a human volunteer's CT scan.

RESULTS

The standard deviation of the difference between the manually measured and Gatortail-based radii in the 3D physical phantom was 0.074 mm (0.087 in-plane pixel units for image voxels of dimension 0.85 × 0.85 × 1.0 mm) over the 64 branches, representing vessel diameters ranging from 1.2 to 7 mm. The linear regression fit gave a slope of 1.056 and an R² value of 0.989. These three metrics reflect superior agreement of the radii estimates relative to previously published results over all sizes tested. Sizing via matched Gaussian filters resulted in size underestimates of >33% over all three test vessels, while the tubularity-metric matching exhibited a sizing uncertainty of >50%. In the human chest CT data set, the vessel voxel intensity profiles with and without branch model optimization showed excellent agreement and improvement in the objective measure of image similarity.

CONCLUSIONS

Gatortail has been demonstrated to be an automated, objective, accurate and robust method for sizing of vessels in 3D non-invasively from chest CT scans. We anticipate that Gatortail, an image-based approach to automatically compute estimates of blood vessel radii and trajectories from 3D medical images, will facilitate future quantitative evaluation of vascular response to disease and environmental insult and improve understanding of the biological mechanisms underlying vascular disease processes.

Tomographic in vivo microscopy for the study of lung physiology at the alveolar level

Lungs represent the essential part of the mammalian respiratory system, which is reflected in the fact that lung failure still is one of the leading causes of morbidity and mortality worldwide. Establishing the connection between macroscopic observations of inspiration and expiration and the processes taking place at the microscopic scale remains crucial to understand fundamental physiological and pathological processes. Here we demonstrate for the first time in vivo synchrotron-based tomographic imaging of lungs with pixel sizes down to a micrometer, enabling first insights into high-resolution lung structure. We report the methodological ability to study lung inflation patterns at the alveolar scale and its potential in resolving still open questions in lung physiology. As a first application, we identified heterogeneous distension patterns at the alveolar level and assessed first comparisons of lungs between the in vivo and immediate post mortem states.

Pulmonary Lobe Segmentation With Probabilistic Segmentation of the Fissures and a Groupwise Fissure Prior

A fully automated, unsupervised lobe segmentation algorithm is presented based on a probabilistic segmentation of the fissures and the simultaneous construction of a populationmodel of the fissures. A two-class probabilistic segmentation segments the lung into candidate fissure voxels and the surrounding parenchyma. This was combined with anatomical information and a groupwise fissure prior to drive non-parametric surface fitting to obtain the final segmentation. The performance of our fissure segmentation was validated on 30 patients from the chronic obstructive pulmonary disease COPDGene cohort, achieving a high median F₁ -score of 0.90 and showed general insensitivity to filter parameters. We evaluated our lobe segmentation algorithm on the Lobe and Lung Analysis 2011 dataset, which contains 55 cases at varying levels of pathology. We achieved the highest score of 0.884 of the automated algorithms. Our method was further tested quantitatively and qualitatively on 80 patients from the COPDgene study at varying levels of functional impairment. Accurate segmentation of the lobes is shown at various degrees of fissure incompleteness for 96% of all cases. We also show the utility of including a groupwise prior in segmenting the lobes in regions of grossly incomplete fissures.

Pleural Adhesion Assessment as a Predictor for Pneumothorax after Endobronchial Valve Treatment

BACKGROUND

Pneumothorax after bronchoscopic lung volume reduction using one-way endobronchial valves (EBVs) in patients with advanced emphysema occurs in approximately 20% of patients. It is not well known which factors predict the development of pneumothorax.

OBJECTIVE

To assess whether pleural adhesions on pretreatment high-resolution computed tomography (HRCT) scans are associated with pneumothorax occurrence after EBV treatment.

METHODS

HRCT scan analyses were performed on all patients who received EBV treatment in a randomized controlled trial. Three blinded readers scored adhesions by number and by measuring the longest axis of each pleural adhesion in the treated lung. The Pleural Adhesion Score (PAS) was calculated by adding 1 point for each small pleural lesion (<1 mm), 5 points for each medium-sized lesion (1-5 mm), and 10 points for each large lesion (>5 mm).

RESULTS

The HRCT scans of 64 treated patients were assessed, of whom 14 developed pneumothorax. Patients who developed pneumothorax had a higher median number of pleural adhesions, 2.7 (IQR 1.9-4) compared to 1.7 (1-2.7) adhesions in the group without pneumothorax (p < 0.01). The PAS in the group with pneumothorax was higher compared to that in the group without: 14.3 (12.4-24.1) versus 6.7 (3.7-11.2) (p < 0.01). A threshold PAS of ≥12 was associated with a higher risk of pneumothorax (OR 13.0, 95% CI 3.1-54.9). A score <12 did not rule out the occurrence of pneumothorax.

CONCLUSION

A higher number of pleural adhesions on HRCT with a subsequent higher PAS in the treated lung is associated with a higher occurrence of pneumothorax after EBV treatment.

CT-Based Local Distribution Metric Improves Characterization of COPD

Parametric response mapping (PRM) of paired CT lung images has been shown to improve the phenotyping of COPD by allowing for the visualization and quantification of non-emphysematous air trapping component, referred to as functional small airways disease (fSAD). Although promising, large variability in the standard method for analyzing PRMfSAD has been observed. We postulate that representing the 3D PRMfSAD data as a single scalar quantity (relative volume of PRMfSAD) oversimplifies the original 3D data, limiting its potential to detect the subtle progression of COPD as well as varying subtypes. In this study, we propose a new approach to analyze PRM. Based on topological techniques, we generate 3D maps of local topological features from 3D PRMfSAD classification maps. We found that the surface area of fSAD (SfSAD) was the most robust and significant independent indicator of clinically meaningful measures of COPD. We also confirmed by micro-CT of human lung specimens that structural differences are associated with unique SfSAD patterns, and demonstrated longitudinal feature alterations occurred with worsening pulmonary function independent of an increase in disease extent. These findings suggest that our technique captures additional COPD characteristics, which may provide important opportunities for improved diagnosis of COPD patients.