Statistical deformation reconstruction using multi-organ shape features for pancreatic cancer localization

Dimension reduction and outlier detection of 3-D shapes derived from multi-organ CT images

BACKGROUND

Unsupervised clustering and outlier detection are important in medical research to understand the distributional composition of a collective of patients. A number of clustering methods exist, also for high-dimensional data after dimension reduction. Clustering and outlier detection may, however, become less robust or contradictory if multiple high-dimensional data sets per patient exist. Such a scenario is given when the focus is on 3-D data of multiple organs per patient, and a high-dimensional feature matrix per organ is extracted.

METHODS

We use principal component analysis (PCA), t-distributed stochastic neighbor embedding (t-SNE) and multiple co-inertia analysis (MCIA) combined with bagplots to study the distribution of multi-organ 3-D data taken by computed tomography scans. After point-set registration of multiple organs from two public data sets, multiple hundred shape features are extracted per organ. While PCA and t-SNE can only be applied to each organ individually, MCIA can project the data of all organs into the same low-dimensional space.

RESULTS

MCIA is the only approach, here, with which data of all organs can be projected into the same low-dimensional space. We studied how frequently (i.e., by how many organs) a patient was classified to belong to the inner or outer 50% of the population, or as an outlier. Outliers could only be detected with MCIA and PCA. MCIA and t-SNE were more robust in judging the distributional location of a patient in contrast to PCA.

CONCLUSIONS

MCIA is more appropriate and robust in judging the distributional location of a patient in the case of multiple high-dimensional data sets per patient. It is still recommendable to apply PCA or t-SNE in parallel to MCIA to study the location of individual organs.

Volumetric Imitation Generative Adversarial Networks for Anatomical Human Body Modeling

Volumetric representation is a technique used to express 3D objects in various fields, such as medical applications. On the other hand, tomography images for reconstructing volumetric data have limited utilization because they contain personal information. Existing GAN-based medical image generation techniques can produce virtual tomographic images for volume reconstruction while preserving the patient's privacy. Nevertheless, these images often do not consider vertical correlations between the adjacent slices, leading to erroneous results in 3D reconstruction. Furthermore, while volume generation techniques have been introduced, they often focus on surface modeling, making it challenging to represent the internal anatomical features accurately. This paper proposes volumetric imitation GAN (VI-GAN), which imitates a human anatomical model to generate volumetric data. The primary goal of this model is to capture the attributes and 3D structure, including the external shape, internal slices, and the relationship between the vertical slices of the human anatomical model. The proposed network consists of a generator for feature extraction and up-sampling based on a 3D U-Net and ResNet structure and a 3D-convolution-based LFFB (local feature fusion block). In addition, a discriminator utilizes 3D convolution to evaluate the authenticity of the generated volume compared to the ground truth. VI-GAN also devises reconstruction loss, including feature and similarity losses, to converge the generated volumetric data into a human anatomical model. In this experiment, the CT data of 234 people were used to assess the reliability of the results. When using volume evaluation metrics to measure similarity, VI-GAN generated a volume that realistically represented the human anatomical model compared to existing volume generation methods.

An augmented reality-guided biopsy system using a high-speed motion tracking and real-time registration platform

Biopsies play a crucial role in diagnosis of various diseases including cancers. In this study, we developed an augmented reality (AR) system to improve biopsy procedures and increase targeting accuracy. Our AR-guided biopsy system uses a high-speed motion tracking technology and an AR headset to display a holographic representation of the organ, lesions, and other structures of interest superimposed on real physical objects. The first application of our AR system is prostate biopsy. By incorporating preoperative scans, such as computed tomography (CT) or magnetic resonance imaging (MRI), into real-time ultrasound-guided procedures, this innovative AR-guided system enables clinicians to see the lesion as well as the organs in real time. With the enhanced visualization of the prostate, lesion, and surrounding organs, surgeons can perform prostate biopsies with an increased accuracy. Our AR-guided biopsy system yielded an average targeting accuracy of 2.94 ± 1.04 mm and can be applied for real-time guidance of prostate biopsy as well as other biopsy procedures.

Liver shape analysis using statistical parametric maps at population scale

BACKGROUND

Morphometric image analysis enables the quantification of differences in the shape and size of organs between individuals.

METHODS

Here we have applied morphometric methods to the study of the liver by constructing surface meshes from liver segmentations from abdominal MRI images in 33,434 participants in the UK Biobank. Based on these three dimensional mesh vertices, we evaluated local shape variations and modelled their association with anthropometric, phenotypic and clinical conditions, including liver disease and type-2 diabetes.

RESULTS

We found that age, body mass index, hepatic fat and iron content, as well as, health traits were significantly associated with regional liver shape and size. Interaction models in groups with specific clinical conditions showed that the presence of type-2 diabetes accelerates age-related changes in the liver, while presence of liver fat further increased shape variations in both type-2 diabetes and liver disease.

CONCLUSIONS

The results suggest that this novel approach may greatly benefit studies aiming at better categorisation of pathologies associated with acute and chronic clinical conditions.

Construction of Shape Atlas for Abdominal Organs using Three-Dimensional Mesh Variational Autoencoder

A model that represents the shapes and positions of organs or skeletal structures with a small number of parameters may be expected to have a wide range of clinical applications, such as radiotherapy and surgical guidance. However, because soft organs vary in shape and position between patients, it is difficult for linear models to reconstruct locally variable shapes, and nonlinear models are prone to overfitting, particularly when the quantity of data is small. The aim of this study was to construct a shape atlas with high accuracy and good generalization performance. We designed a mesh variational autoencoder that can reconstruct both nonlinear shape and position with high accuracy. We validated the trained model for liver meshes of 125 cases, and found that it was possible to reconstruct the positions and shapes with an average accuracy of 4.3 mm for the test data of 19 cases.

Landmark-free Shape Analysis of the Human Duodenum. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023;2023:1-4. [PMID: 38083606 DOI: 10.1109/embc40787.2023.10340464] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

The primary function of the duodenum is to undertake chemical digestion by ensuring that the partially digested food received from the stomach is well-mixed with the enzymes and chemicals secreted into it. However, little is known about the anatomical variations in the shape of the duodenum within humans, and thus the effect of duodenum shape on the flow and mixing occurring within the lumen has not been studied. In this work, a methodology for analyzing shape variations in the normal duodenal anatomy has been developed and applied to a publicly available dataset of abdominal CT images. This method does not require the placement of landmarks as it is based on the underlying tubular 'C' shape of the duodenum. The average duodenal length and radius of this dataset (consisting of 34 subjects) were 212.8 ± 38 mm and 10.8 ± 2.5 mm respectively. A Principal Component Analysis (PCA) was conducted on a sample of 34 duodenums after normalizing their lengths and the first five principal components were found to contribute to 82 % of the total variation. The first shape component (accounting for 42 % of overall variation) consisted of variations in the radius along the duodenum with no deformations normal to the central plane, and the subsequent shape modes consisted of twists in the centerline either in and out of the central plane, and radial variations at either the inlet or outlet. This is the first study to analyze shape variations in the human duodenum and the results can be combined with flow modeling to analyze the effect of shape on the flow and mixing occurring within the duodenum.Clinical relevance- The methods developed in this study can be used by clinicians to diagnose abnormalities in an individual's duodenum shape.

Dynamic multi feature-class Gaussian process models

In model-based medical image analysis, three relevant features are the shape of structures of interest, their relative pose, and image intensity profiles representative of some physical properties. Often, these features are modelled separately through statistical models by decomposing the object's features into a set of basis functions through principal geodesic analysis or principal component analysis. However, analysing articulated objects in an image using independent single object models may lead to large uncertainties and impingement, especially around organ boundaries. Questions that come to mind are the feasibility of building a unique model that combines all three features of interest in the same statistical space, and what advantages can be gained for image analysis. This study presents a statistical modelling method for automatic analysis of shape, pose and intensity features in medical images which we call the Dynamic multi feature-class Gaussian process models (DMFC-GPM). The DMFC-GPM is a Gaussian process (GP)-based model with a shared latent space that encodes linear and non-linear variations. Our method is defined in a continuous domain with a principled way to represent shape, pose and intensity feature-classes in a linear space, based on deformation fields. A deformation field-based metric is adapted in the method for modelling shape and intensity variation as well as for comparing rigid transformations (pose). Moreover, DMFC-GPMs inherit properties intrinsic to GPs including marginalisation and regression. Furthermore, they allow for adding additional pose variability on top of those obtained from the image acquisition process; what we term as permutation modelling. For image analysis tasks using DMFC-GPMs, we adapt Metropolis-Hastings algorithms making the prediction of features fully probabilistic. We validate the method using controlled synthetic data and we perform experiments on bone structures from CT images of the shoulder to illustrate the efficacy of the model for pose and shape prediction. The model performance results suggest that this new modelling paradigm is robust, accurate, accessible, and has potential applications in a multitude of scenarios including the management of musculoskeletal disorders, clinical decision making and image processing.

Evaluation of generalization ability for deep learning-based auto-segmentation accuracy in limited field of view CBCT of male pelvic region

PURPOSE

The aim of this study was to evaluate generalization ability of segmentation accuracy for limited FOV CBCT in the male pelvic region using a full-image CNN. Auto-segmentation accuracy was evaluated using various datasets with different intensity distributions and FOV sizes.

METHODS

A total of 171 CBCT datasets from patients with prostate cancer were enrolled. There were 151, 10, and 10 CBCT datasets acquired from Vero4DRT, TrueBeam STx, and Clinac-iX, respectively. The FOV for Vero4DRT, TrueBeam STx, and Clinac-iX was 20, 26, and 25 cm, respectively. The ROIs, including the bladder, prostate, rectum, and seminal vesicles, were manually delineated. The U² -Net CNN network architecture was used to train the segmentation model. A total of 131 limited FOV CBCT datasets from Vero4DRT were used for training (104 datasets) and validation (27 datasets); thereafter the rest were for testing. The training routine was set to save the best weight values when the DSC in the validation set was maximized. Segmentation accuracy was qualitatively and quantitatively evaluated between the ground truth and predicted ROIs in the different testing datasets.

RESULTS

The mean scores ± standard deviation of visual evaluation for bladder, prostate, rectum, and seminal vesicle in all treatment machines were 1.0 ± 0.7, 1.5 ± 0.6, 1.4 ± 0.6, and 2.1 ± 0.8 points, respectively. The median DSC values for all imaging devices were ≥0.94 for the bladder, 0.84-0.87 for the prostate and rectum, and 0.48-0.69 for the seminal vesicles. Although the DSC values for the bladder and seminal vesicles were significantly different among the three imaging devices, the DSC value of the bladder changed by less than 1% point. The median MSD values for all imaging devices were ≤1.2 mm for the bladder and 1.4-2.2 mm for the prostate, rectum, and seminal vesicles. The MSD values for the seminal vesicles were significantly different between the three imaging devices.

CONCLUSION

The proposed method is effective for testing datasets with different intensity distributions and FOV from training datasets.

Image-to-Graph Convolutional Network for 2D/3D Deformable Model Registration of Low-Contrast Organs

Organ shape reconstruction based on a single-projection image during treatment has wide clinical scope, e.g., in image-guided radiotherapy and surgical guidance. We propose an image-to-graph convolutional network that achieves deformable registration of a three-dimensional (3D) organ mesh for a low-contrast two-dimensional (2D) projection image. This framework enables simultaneous training of two types of transformation: from the 2D projection image to a displacement map, and from the sampled per-vertex feature to a 3D displacement that satisfies the geometrical constraint of the mesh structure. Assuming application to radiation therapy, the 2D/3D deformable registration performance is verified for multiple abdominal organs that have not been targeted to date, i.e., the liver, stomach, duodenum, and kidney, and for pancreatic cancer. The experimental results show shape prediction considering relationships among multiple organs can be used to predict respiratory motion and deformation from digitally reconstructed radiographs with clinically acceptable accuracy.

Image-Guided Percutaneous Ablation for Primary and Metastatic Tumors

Image-guided percutaneous ablation methods have been further developed during the recent two decades and have transformed the minimally invasive and precision features of treatment options targeting primary and metastatic tumors. They work by percutaneously introducing applicators to precisely destroy a tumor and offer much lower risks than conventional methods. There are usually shorter recovery periods, less bleeding, and more preservation of organ parenchyma, expanding the treatment options of patients with cancer who may not be eligible for resection. Image-guided ablation techniques are currently utilized for the treatment of primary and metastatic tumors in various organs including the liver, pancreas, kidneys, thyroid and parathyroid, prostate, lung, bone, and soft tissue. This article provides a brief review of the various imaging modalities and available ablation techniques and discusses their applications and associated complications in various organs.

Gastric deformation models for adaptive radiotherapy: Personalized vs population-based strategy

BACKGROUND AND PURPOSE

To create a library of plans (LoP) for gastric cancer adaptive radiotherapy, accurate predictions of shape changes due to filling variations are essential. The ability of two strategies (personalized and population-based) to predict stomach shape based on filling was evaluated for volunteer and patient data to explore the potential for use in a LoP.

MATERIALS AND METHODS

For 19 healthy volunteers, stomachs were delineated on MRIs with empty (ES), half-full (HFS) and full stomach (FS). For the personalized strategy, a deformation vector field from HFS to corresponding ES was acquired and extrapolated to predict FS. For the population-based strategy, the average deformation vectors from HFS to FS of 18 volunteers were applied to the HFS of the remaining volunteer to predict FS (leave-one-out principle); thus, predictions were made for each volunteer. Reversed processes were performed to predict ES. To validate, for seven gastric cancer patients, the volunteer population-based model was applied to their pre-treatment CT to predict stomach shape on 2-3 repeat CTs. For all predictions, volume was made equal to true stomach volume.

RESULTS

FS predictions were satisfactory, with median Dice similarity coefficient (mDSC) of 0.91 (population-based) and 0.89 (personalized). ES predictions were poorer: mDSC = 0.82 for population-based; personalized strategy yielded unachievable volumes. Population-based shape predictions (both ES and FS) were comparable between patients (mDSC = 0.87) and volunteers (0.88).

CONCLUSION

The population-based model outperformed the personalized model and demonstrated its ability in predicting filling-dependent stomach shape changes and, therefore, its potential for use in a gastric cancer LoP.

Shape Reconstruction for Abdominal Organs based on a Graph Convolutional Network

Computed tomography and magnetic resonance imaging produce high-resolution images; however, during surgery or radiotherapy, only low-resolution cone-beam CT and low-dimensional X-ray images can be obtained. Furthermore, because the duodenum and stomach are filled with air, even in high-resolution CT images, it is hard to accurately segment their contours. In this paper, we propose a method that is based on a graph convolutional network (GCN) to reconstruct organs that are hard to detect in medical images. The method uses surrounding detectable-organ features to determine the shape and location of the target organ and learns mesh deformation parameters, which are applied to a target organ template. The role of the template is to establish an initial topological structure for the target organ. We conducted experiments with both single and multiple organ meshes to verify the performance of our proposed method.

Synthetic CT-aided multiorgan segmentation for CBCT-guided adaptive pancreatic radiotherapy

PURPOSE

The delineation of organs at risk (OARs) is fundamental to cone-beam CT (CBCT)-based adaptive radiotherapy treatment planning, but is time consuming, labor intensive, and subject to interoperator variability. We investigated a deep learning-based rapid multiorgan delineation method for use in CBCT-guided adaptive pancreatic radiotherapy.

METHODS

To improve the accuracy of OAR delineation, two innovative solutions have been proposed in this study. First, instead of directly segmenting organs on CBCT images, a pretrained cycle-consistent generative adversarial network (cycleGAN) was applied to generating synthetic CT images given CBCT images. Second, an advanced deep learning model called mask-scoring regional convolutional neural network (MS R-CNN) was applied on those synthetic CT to detect the positions and shapes of multiple organs simultaneously for final segmentation. The OAR contours delineated by the proposed method were validated and compared with expert-drawn contours for geometric agreement using the Dice similarity coefficient (DSC), 95th percentile Hausdorff distance (HD95), mean surface distance (MSD), and residual mean square distance (RMS).

RESULTS

Across eight abdominal OARs including duodenum, large bowel, small bowel, left and right kidneys, liver, spinal cord, and stomach, the geometric comparisons between automated and expert contours are as follows: 0.92 (0.89-0.97) mean DSC, 2.90 mm (1.63-4.19 mm) mean HD95, 0.89 mm (0.61-1.36 mm) mean MSD, and 1.43 mm (0.90-2.10 mm) mean RMS. Compared to the competing methods, our proposed method had significant improvements (p < 0.05) in all the metrics for all the eight organs. Once the model was trained, the contours of eight OARs can be obtained on the order of seconds.

CONCLUSIONS

We demonstrated the feasibility of a synthetic CT-aided deep learning framework for automated delineation of multiple OARs on CBCT. The proposed method could be implemented in the setting of pancreatic adaptive radiotherapy to rapidly contour OARs with high accuracy.

Improvement of Image Quality of Cone-beam CT Images by Three-dimensional Generative Adversarial Network

Artifacts and defects in Cone-beam Computed Tomography (CBCT) images are a problem in radiotherapy and surgical procedures. Unsupervised learning-based image translation techniques have been studied to improve the image quality of head and neck CBCT images, but there have been few studies on improving the image quality of abdominal CBCT images, which are strongly affected by organ deformation due to posture and breathing. In this study, we propose a method for improving the image quality of abdominal CBCT images by translating the numerical values to the values of corresponding paired CT images using an unsupervised CycleGAN framework. This method preserves anatomical structure through adversarial learning that translates voxel values according to corresponding regions between CBCT and CT images of the same case. The image translation model was trained on 68 CT-CBCT datasets and then applied to 8 test datasets, and the effectiveness of the proposed method for improving the image quality of CBCT images was confirmed.

Deformation analysis of surface and bronchial structures in intraoperative pneumothorax using deformable mesh registration

The positions of nodules can change because of intraoperative lung deflation, and the modeling of pneumothorax-associated deformation remains a challenging issue for intraoperative tumor localization. In this study, we introduce spatial and geometric analysis methods for inflated/deflated lungs and discuss heterogeneity in pneumothorax-associated lung deformation. Contrast-enhanced CT images simulating intraoperative conditions were acquired from live Beagle dogs. The images contain the overall shape of the lungs, including all lobes and internal bronchial structures, and were analyzed to provide a statistical deformation model that could be used as prior knowledge to predict pneumothorax. To address the difficulties of mapping pneumothorax CT images with topological changes and CT intensity shifts, we designed deformable mesh registration techniques for mixed data structures including the lobe surfaces and the bronchial centerlines. Three global-to-local registration steps were performed under the constraint that the deformation was spatially continuous and smooth, while matching visible bronchial tree structures as much as possible. The developed framework achieved stable registration with a Hausdorff distance of less than 1 mm and a target registration error of less than 5 mm, and visualized deformation fields that demonstrate per-lobe contractions and rotations with high variability between subjects. The deformation analysis results show that the strain of lung parenchyma was 35% higher than that of bronchi, and that deformation in the deflated lung is heterogeneous.

Statistical shape model-based planning organ-at-risk volume: application to pancreatic cancer patients

PURPOSE

To introduce the concept of statistical shape model (SSM)-based planning organ-at-risk volume (sPRV) for pancreatic cancer patients.

METHODS

A total of 120 pancreatic cancer patients were enrolled in this study. After correcting inter-patient variations in the centroid position of the planning target volume (PTV), four different SSMs were constructed by registering a deformable template model to an individual model for the stomach and duodenum. The sPRV, which focused on the following different components of the inter-patient variations, was then created: Scenario A: shape, rotational angle, volume, and centroid position; Scenario B: shape, rotational angle, and volume; Scenario C: shape and rotational angle; and Scenario D: shape. The conventional PRV (cPRV) was created by adding an isotropic margin R (3-15 mm) to the mean shape model. The corresponding sPRV was created from the SSM until the volume difference between the cPRV and sPRV was less than 1%. Thereafter, we computed the overlapping volume between the PTV and cPRV (OL_c) or sPRV (OL_s) in each patient. OL_s being larger than OL_c implies that the local shape variations in the corresponding OAR close to the PTV were large. Therefore, OL_s/OL_c was calculated in each patient for each R-value, and the median value of OL_s/OL_c was regarded as a surrogate for plan quality for each R-value.

RESULTS

For R = 3 and 5 mm, OL_s/OL_c exceeded 1 for the stomach and duodenum in all scenarios, with a maximum OL_s/OL_c of 1.21. This indicates that smaller isotropic margins did not sufficiently account for the local shape changes close to the PTV.

CONCLUSIONS

Our results indicated that, in contrast to conventional PRV, SSM-based PRVs, which account for local shape changes, would result in better dose sparing for the stomach and duodenum in pancreatic cancer patients.