Technical Note: Automatic segmentation of CT images for ventral body composition analysis

Accelerated muscle mass estimation from CT images through transfer learning

BACKGROUND

The cost of labeling to collect training data sets using deep learning is especially high in medical applications compared to other fields. Furthermore, due to variances in images depending on the computed tomography (CT) devices, a deep learning based segmentation model trained with a certain device often does not work with images from a different device.

METHODS

In this study, we propose an efficient learning strategy for deep learning models in medical image segmentation. We aim to overcome the difficulties of segmentation in CT images by training a VNet segmentation model which enables rapid labeling of organs in CT images with the model obtained by transfer learning using a small number of manually labeled images, called SEED images. We established a process for generating SEED images and conducting transfer learning a model. We evaluate the performance of various segmentation models such as vanilla UNet, UNETR, Swin-UNETR and VNet. Furthermore, assuming a scenario that a model is repeatedly trained with CT images collected from multiple devices, in which is catastrophic forgetting often occurs, we examine if the performance of our model degrades.

RESULTS

We show that transfer learning can train a model that does a good job of segmenting muscles with a small number of images. In addition, it was confirmed that VNet shows better performance when comparing the performance of existing semi-automated segmentation tools and other deep learning networks to muscle and liver segmentation tasks. Additionally, we confirmed that VNet is the most robust model to deal with catastrophic forgetting problems.

CONCLUSION

In the 2D CT image segmentation task, we confirmed that the CNN-based network shows better performance than the existing semi-automatic segmentation tool or latest transformer-based networks.

A systematic review of automated segmentation of 3D computed-tomography scans for volumetric body composition analysis

Automated computed tomography (CT) scan segmentation (labelling of pixels according to tissue type) is now possible. This technique is being adapted to achieve three-dimensional (3D) segmentation of CT scans, opposed to single L3-slice alone. This systematic review evaluates feasibility and accuracy of automated segmentation of 3D CT scans for volumetric body composition (BC) analysis, as well as current limitations and pitfalls clinicians and researchers should be aware of. OVID Medline, Embase and grey literature databases up to October 2021 were searched. Original studies investigating automated skeletal muscle, visceral and subcutaneous AT segmentation from CT were included. Seven of the 92 studies met inclusion criteria. Variation existed in expertise and numbers of humans performing ground-truth segmentations used to train algorithms. There was heterogeneity in patient characteristics, pathology and CT phases that segmentation algorithms were developed upon. Reporting of anatomical CT coverage varied, with confusing terminology. Six studies covered volumetric regional slabs rather than the whole body. One study stated the use of whole-body CT, but it was not clear whether this truly meant head-to-fingertip-to-toe. Two studies used conventional computer algorithms. The latter five used deep learning (DL), an artificial intelligence technique where algorithms are similarly organized to brain neuronal pathways. Six of seven reported excellent segmentation performance (Dice similarity coefficients > 0.9 per tissue). Internal testing on unseen scans was performed for only four of seven algorithms, whilst only three were tested externally. Trained DL algorithms achieved full CT segmentation in 12 to 75 s versus 25 min for non-DL techniques. DL enables opportunistic, rapid and automated volumetric BC analysis of CT performed for clinical indications. However, most CT scans do not cover head-to-fingertip-to-toe; further research must validate using common CT regions to estimate true whole-body BC, with direct comparison to single lumbar slice. Due to successes of DL, we expect progressive numbers of algorithms to materialize in addition to the seven discussed in this paper. Researchers and clinicians in the field of BC must therefore be aware of pitfalls. High Dice similarity coefficients do not inform the degree to which BC tissues may be under- or overestimated and nor does it inform on algorithm precision. Consensus is needed to define accuracy and precision standards for ground-truth labelling. Creation of a large international, multicentre common CT dataset with BC ground-truth labels from multiple experts could be a robust solution.

Abdomen CT multi-organ segmentation using token-based MLP-Mixer

BACKGROUND

Manual contouring is very labor-intensive, time-consuming, and subject to intra- and inter-observer variability. An automated deep learning approach to fast and accurate contouring and segmentation is desirable during radiotherapy treatment planning.

PURPOSE

This work investigates an efficient deep-learning-based segmentation algorithm in abdomen computed tomography (CT) to facilitate radiation treatment planning.

METHODS

In this work, we propose a novel deep-learning model utilizing U-shaped multi-layer perceptron mixer (MLP-Mixer) and convolutional neural network (CNN) for multi-organ segmentation in abdomen CT images. The proposed model has a similar structure to V-net, while a proposed MLP-Convolutional block replaces each convolutional block. The MLP-Convolutional block consists of three components: an early convolutional block for local features extraction and feature resampling, a token-based MLP-Mixer layer for capturing global features with high efficiency, and a token projector for pixel-level detail recovery. We evaluate our proposed network using: (1) an institutional dataset with 60 patient cases and (2) a public dataset (BCTV) with 30 patient cases. The network performance was quantitatively evaluated in three domains: (1) volume similarity between the ground truth contours and the network predictions using the Dice score coefficient (DSC), sensitivity, and precision; (2) surface similarity using Hausdorff distance (HD), mean surface distance (MSD) and residual mean square distance (RMS); and (3) the computational complexity reported by the number of network parameters, training time, and inference time. The performance of the proposed network is compared with other state-of-the-art networks.

RESULTS

In the institutional dataset, the proposed network achieved the following volume similarity measures when averaged over all organs: DSC = 0.912, sensitivity = 0.917, precision = 0.917, average surface similarities were HD = 11.95 mm, MSD = 1.90 mm, RMS = 3.86 mm. The proposed network achieved DSC = 0.786 and HD = 9.04 mm on the public dataset. The network also shows statistically significant improvement, which is evaluated by a two-tailed Wilcoxon Mann-Whitney U test, on right lung (MSD where the maximum p-value is 0.001), spinal cord (sensitivity, precision, HD, RMSD where p-value ranges from 0.001 to 0.039), and stomach (DSC where the maximum p-value is 0.01) over all other competing networks. On the public dataset, the network report statistically significant improvement, which is shown by the Wilcoxon Mann-Whitney test, on pancreas (HD where the maximum p-value is 0.006), left (HD where the maximum p-value is 0.022) and right adrenal glands (DSC where the maximum p-value is 0.026). In both datasets, the proposed method can generate contours in less than 5 s. Overall, the proposed MLP-Vnet demonstrates comparable or better performance than competing methods with much lower memory complexity and higher speed.

CONCLUSIONS

The proposed MLP-Vnet demonstrates superior segmentation performance, in terms of accuracy and efficiency, relative to state-of-the-art methods. This reliable and efficient method demonstrates potential to streamline clinical workflows in abdominal radiotherapy, which may be especially important for online adaptive treatments.

Estimating 3-D whole-body composition from a chest CT scan

BACKGROUND

Estimating whole-body composition from limited region-computed tomography (CT) scans has many potential applications in clinical medicine; however, it is challenging.

PURPOSE

To investigate if whole-body composition based on several tissue types (visceral adipose tissue [VAT], subcutaneous adipose tissue [SAT], intermuscular adipose tissue [IMAT], skeletal muscle [SM], and bone) can be reliably estimated from a chest CT scan only.

METHODS

A cohort of 97 lung cancer subjects who underwent both chest CT scans and whole-body positron emission tomography-CT scans at our institution were collected. We used our in-house software to automatically segment and quantify VAT, SAT, IMAT, SM, and bone on the CT images. The field-of-views of the chest CT scans and the whole-body CT scans were standardized, namely, from vertebra T1 to L1 and from C1 to the bottom of the pelvis, respectively. Multivariate linear regression was used to develop the computer models for estimating the volumes of whole-body tissues from chest CT scans. Subject demographics (e.g., gender and age) and lung volume were included in the modeling analysis. Ten-fold cross-validation was used to validate the performance of the prediction models. Mean absolute difference (MAD) and R-squared (R2 ) were used as the performance metrics to assess the model performance.

RESULTS

The R2 values when estimating volumes of whole-body SAT, VAT, IMAT, total fat, SM, and bone from the regular chest CT scans were 0.901, 0.929, 0.900, 0.933, 0.928, and 0.918, respectively. The corresponding MADs (percentage difference) were 1.44 ± 1.21 L (12.21% ± 11.70%), 0.63 ± 0.49 L (29.68% ± 61.99%), 0.12 ± 0.09 L (16.20% ± 18.42%), 1.65 ± 1.40 L (10.43% ± 10.79%), 0.71 ± 0.68 L (5.14% ± 4.75%), and 0.17 ± 0.15 L (4.32% ± 3.38%), respectively.

CONCLUSION

Our algorithm shows promise in its ability to estimate whole-body compositions from chest CT scans. Body composition measures based on chest CT scans are more accurate than those based on vertebra third lumbar.

AI-Supported Comprehensive Detection and Quantification of Biomarkers of Subclinical Widespread Diseases at Chest CT for Preventive Medicine

Automated image analysis plays an increasing role in radiology in detecting and quantifying image features outside of the perception of human eyes. Common AI-based approaches address a single medical problem, although patients often present with multiple interacting, frequently subclinical medical conditions. A holistic imaging diagnostics tool based on artificial intelligence (AI) has the potential of providing an overview of multi-system comorbidities within a single workflow. An interdisciplinary, multicentric team of medical experts and computer scientists designed a pipeline, comprising AI-based tools for the automated detection, quantification and characterization of the most common pulmonary, metabolic, cardiovascular and musculoskeletal comorbidities in chest computed tomography (CT). To provide a comprehensive evaluation of each patient, a multidimensional workflow was established with algorithms operating synchronously on a decentralized Joined Imaging Platform (JIP). The results of each patient are transferred to a dedicated database and summarized as a structured report with reference to available reference values and annotated sample images of detected pathologies. Hence, this tool allows for the comprehensive, large-scale analysis of imaging-biomarkers of comorbidities in chest CT, first in science and then in clinical routine. Moreover, this tool accommodates the quantitative analysis and classification of each pathology, providing integral diagnostic and prognostic value, and subsequently leading to improved preventive patient care and further possibilities for future studies.

Unsupervised Domain Adaptation for Vertebrae Detection and Identification in 3D CT Volumes Using a Domain Sanity Loss

A variety of medical computer vision applications analyze 2D slices of computed tomography (CT) scans, whereas axial slices from the body trunk region are usually identified based on their relative position to the spine. A limitation of such systems is that either the correct slices must be extracted manually or labels of the vertebrae are required for each CT scan to develop an automated extraction system. In this paper, we propose an unsupervised domain adaptation (UDA) approach for vertebrae detection and identification based on a novel Domain Sanity Loss (DSL) function. With UDA the model’s knowledge learned on a publicly available (source) data set can be transferred to the target domain without using target labels, where the target domain is defined by the specific setup (CT modality, study protocols, applied pre- and processing) at the point of use (e.g., a specific clinic with its specific CT study protocols). With our approach, a model is trained on the source and target data set in parallel. The model optimizes a supervised loss for labeled samples from the source domain and the DSL loss function based on domain-specific “sanity checks” for samples from the unlabeled target domain. Without using labels from the target domain, we are able to identify vertebra centroids with an accuracy of 72.8%. By adding only ten target labels during training the accuracy increases to 89.2%, which is on par with the current state-of-the-art for full supervised learning, while using about 20 times less labels. Thus, our model can be used to extract 2D slices from 3D CT scans on arbitrary data sets fully automatically without requiring an extensive labeling effort, contributing to the clinical adoption of medical imaging by hospitals.

Deep learning for abdominal adipose tissue segmentation with few labelled samples

PURPOSE

Fully automated abdominal adipose tissue segmentation from computed tomography (CT) scans plays an important role in biomedical diagnoses and prognoses. However, to identify and segment subcutaneous adipose tissue (SAT) and visceral adipose tissue (VAT) in the abdominal region, the traditional routine process used in clinical practise is unattractive, expensive, time-consuming and leads to false segmentation. To address this challenge, this paper introduces and develops an effective global-anatomy-level convolutional neural network (ConvNet) automated segmentation of abdominal adipose tissue from CT scans termed EFNet to accommodate multistage semantic segmentation and high similarity intensity characteristics of the two classes (VAT and SAT) in the abdominal region.

METHODS

EFNet consists of three pathways: (1) The first pathway is the max unpooling operator, which was used to reduce computational consumption. (2) The second pathway is concatenation, which was applied to recover the shape segmentation results. (3) The third pathway is anatomy pyramid pooling, which was adopted to obtain fine-grained features. The usable anatomical information was encoded in the output of EFNet and allowed for the control of the density of the fine-grained features.

RESULTS

We formulated an end-to-end manner for the learning process of EFNet, where the representation features can be jointly learned through a mixed feature fusion layer. We immensely evaluated our model on different datasets and compared it to existing deep learning networks. Our proposed model called EFNet outperformed other state-of-the-art models on the segmentation results and demonstrated tremendous performances for abdominal adipose tissue segmentation.

CONCLUSION

EFNet is extremely fast with remarkable performance for fully automated segmentation of the VAT and SAT in abdominal adipose tissue from CT scans. The proposed method demonstrates a strength ability for automated detection and segmentation of abdominal adipose tissue in clinical practise.

A review of deep learning based methods for medical image multi-organ segmentation

Deep learning has revolutionized image processing and achieved the-state-of-art performance in many medical image segmentation tasks. Many deep learning-based methods have been published to segment different parts of the body for different medical applications. It is necessary to summarize the current state of development for deep learning in the field of medical image segmentation. In this paper, we aim to provide a comprehensive review with a focus on multi-organ image segmentation, which is crucial for radiotherapy where the tumor and organs-at-risk need to be contoured for treatment planning. We grouped the surveyed methods into two broad categories which are 'pixel-wise classification' and 'end-to-end segmentation'. Each category was divided into subgroups according to their network design. For each type, we listed the surveyed works, highlighted important contributions and identified specific challenges. Following the detailed review, we discussed the achievements, shortcomings and future potentials of each category. To enable direct comparison, we listed the performance of the surveyed works that used thoracic and head-and-neck benchmark datasets.