Medially constrained deformable modeling for segmentation of branching medial structures: Application to aortic valve segmentation and morphometry

Automatic segmentation of medial temporal lobe subregions in multi-scanner, multi-modality MRI of variable quality

Background

Volumetry of subregions in the medial temporal lobe (MTL) computed from automatic segmentation in MRI can track neurodegeneration in Alzheimer's disease. However, image quality may vary in MRI. Poor quality MR images can lead to unreliable segmentation of MTL subregions. Considering that different MRI contrast mechanisms and field strengths (jointly referred to as "modalities" here) offer distinct advantages in imaging different parts of the MTL, we developed a muti-modality segmentation model using both 7 tesla (7T) and 3 tesla (3T) structural MRI to obtain robust segmentation in poor-quality images.

Method

MRI modalities including 3T T1-weighted, 3T T2-weighted, 7T T1-weighted and 7T T2-weighted (7T-T2w) of 197 participants were collected from a longitudinal aging study at the Penn Alzheimer's Disease Research Center. Among them, 7T-T2w was used as the primary modality, and all other modalities were rigidly registered to the 7T-T2w. A model derived from nnU-Net took these registered modalities as input and outputted subregion segmentation in 7T-T2w space. 7T-T2w images most of which had high quality from 25 selected training participants were manually segmented to train the multi-modality model. Modality augmentation, which randomly replaced certain modalities with Gaussian noise, was applied during training to guide the model to extract information from all modalities. To compare our proposed model with a baseline single-modality model in the full dataset with mixed high/poor image quality, we evaluated the ability of derived volume/thickness measures to discriminate Amyloid+ mild cognitive impairment (A+MCI) and Amyloid- cognitively unimpaired (A-CU) groups, as well as the stability of these measurements in longitudinal data.

Results

The multi-modality model delivered good performance regardless of 7T-T2w quality, while the single-modality model under-segmented subregions in poor-quality images. The multi-modality model generally demonstrated stronger discrimination of A+MCI versus A-CU. Intra-class correlation and Bland-Altman plots demonstrate that the multi-modality model had higher longitudinal segmentation consistency in all subregions while the single-modality model had low consistency in poor-quality images.

Conclusion

The multi-modality MRI segmentation model provides an improved biomarker for neurodegeneration in the MTL that is robust to image quality. It also provides a framework for other studies which may benefit from multimodal imaging.

Open-source graphical user interface for the creation of synthetic skeletons for medical image analysis

Purpose

Deformable medial modeling is an inverse skeletonization approach to representing anatomy in medical images, which can be used for statistical shape analysis and assessment of patient-specific anatomical features such as locally varying thickness. It involves deforming a pre-defined synthetic skeleton, or template, to anatomical structures of the same class. The lack of software for creating such skeletons has been a limitation to more widespread use of deformable medial modeling. Therefore, the objective of this work is to present an open-source user interface (UI) for the creation of synthetic skeletons for a range of medial modeling applications in medical imaging.

Approach

A UI for interactive design of synthetic skeletons was implemented in 3D Slicer, an open-source medical image analysis application. The steps in synthetic skeleton design include importation and skeletonization of a 3D segmentation, followed by interactive 3D point placement and triangulation of the medial surface such that the desired branching configuration of the anatomical structure's medial axis is achieved. Synthetic skeleton design was evaluated in five clinical applications. Compatibility of the synthetic skeletons with open-source software for deformable medial modeling was tested, and representational accuracy of the deformed medial models was evaluated.

Results

Three users designed synthetic skeletons of anatomies with various topologies: the placenta, aortic root wall, mitral valve, cardiac ventricles, and the uterus. The skeletons were compatible with skeleton-first and boundary-first software for deformable medial modeling. The fitted medial models achieved good representational accuracy with respect to the 3D segmentations from which the synthetic skeletons were generated.

Conclusions

Synthetic skeleton design has been a practical challenge in leveraging deformable medial modeling for new clinical applications. This work demonstrates an open-source UI for user-friendly design of synthetic skeletons for anatomies with a wide range of topologies.

Strain estimation in aortic roots from 4D echocardiographic images using medial modeling and deformable registration

Even though the central role of mechanics in the cardiovascular system is widely recognized, estimating mechanical deformation and strains in-vivo remains an ongoing practical challenge. Herein, we present a semi-automated framework to estimate strains from four-dimensional (4D) echocardiographic images and apply it to the aortic roots of patients with normal trileaflet aortic valves (TAV) and congenital bicuspid aortic valves (BAV). The method is based on fully nonlinear shell-based kinematics, which divides the strains into in-plane (shear and dilatational) and out-of-plane components. The results indicate that, even for size-matched non-aneurysmal aortic roots, BAV patients experience larger regional shear strains in their aortic roots. This elevated strains might be a contributing factor to the higher risk of aneurysm development in BAV patients. The proposed framework is openly available and applicable to any tubular structures.

Skeletons, Object Shape, Statistics

Objects and object complexes in 3D, as well as those in 2D, have many possible representations. Among them skeletal representations have special advantages and some limitations. For the special form of skeletal representation called "s-reps," these advantages include strong suitability for representing slabular object populations and statistical applications on these populations. Accomplishing these statistical applications is best if one recognizes that s-reps live on a curved shape space. Here we will lay out the definition of s-reps, their advantages and limitations, their mathematical properties, methods for fitting s-reps to single- and multi-object boundaries, methods for measuring the statistics of these object and multi-object representations, and examples of such applications involving statistics. While the basic theory, ideas, and programs for the methods are described in this paper and while many applications with evaluations have been produced, there remain many interesting open opportunities for research on comparisons to other shape representations, new areas of application and further methodological developments, many of which are explicitly discussed here.

Patient-Specific Quantification of Normal and Bicuspid Aortic Valve Leaflet Deformations from Clinically Derived Images

The clinical benefit of patient-specific modeling of heart valve disease remains an unrealized goal, often a result of our limited understanding of the in vivo milieu. This is particularly true in assessing bicuspid aortic valve (BAV) disease, the most common cardiac congenital defect in humans, which leads to premature and severe aortic stenosis or insufficiency (AS/AI). However, assessment of BAV risk for AS/AI on a patient-specific basis is hampered by the substantial degree of anatomic and functional variations that remain largely unknown. The present study was undertaken to utilize a noninvasive computational pipeline ( https://doi.org/10.1002/cnm.3142 ) that directly yields local heart valve leaflet deformation information using patient-specific real-time three-dimensional echocardiographic imaging (rt-3DE) data. Imaging data was collected for patients with normal tricuspid aortic valve (TAV, [Formula: see text]) and those with BAV ([Formula: see text] with fused left and right coronary leaflets and [Formula: see text] with fused right and non-coronary leaflets), from which the medial surface of each leaflet was extracted. The resulting deformation analysis resulted in, for the first time, quantified differences between the in vivo functional deformations of the TAV and BAV leaflets. Our approach was able to capture the complex, heterogeneous surface deformation fields in both TAV and BAV leaflets. We were able to identify and quantify differences in stretch patterns between leaflet types, and found in particular that stretches experienced by BAV leaflets during closure differ from those of TAV leaflets in terms of both heterogeneity as well as overall magnitude. Deformation is a key parameter in the clinical assessment of valvular function, and serves as a direct means to determine regional variations in structure and function. This study is an essential step toward patient-specific assessment of BAV based on correlating leaflet deformation and AS/AI progression, as it provides a means for assessing patient-specific stretch patterns.

Local aortic aneurysm wall expansion measured with automated image analysis

Background

Assessment of regional aortic wall deformation (RAWD) might better predict for abdominal aortic aneurysm (AAA) rupture than the maximal aortic diameter or growth rate. Using sequential computed tomography angiograms (CTAs), we developed a streamlined, semiautomated method of computing RAWD using deformable image registration (dirRAWD).

Methods

Paired sequential CTAs performed 1 to 2 years apart of 15 patients with AAAs of various shapes and sizes were selected. Using each patient’s initial CTA, the luminal and aortic wall surfaces were segmented both manually and semiautomatically. Next, the same patient’s follow-up CTA was aligned with the first using automated rigid image registration. Deformable image registration was then used to calculate the local aneurysm wall expansion between the sequential scans (dirRAWD). To measure technique accuracy, the deformable registration results were compared with the local displacement of anatomic landmarks (fiducial markers), such as the origin of the inferior mesenteric artery and/or aortic wall calcifications. Additionally, for each patient, the maximal RAWD was manually measured for each aneurysm and was compared with the dirRAWD at the same location.

Results

The technique was successful in all patients. The mean landmark displacement error was 0.59 ± 0.93 mm with no difference between true landmark displacement and deformable registration landmark displacement by Wilcoxon rank sum test (P = .39). The absolute difference between the manually measured maximal RAWD and dirRAWD was 0.27 ± 0.23 mm, with a relative difference of 7.9% and no difference using the Wilcoxon rank sum test (P = .69). No differences were found in the maximal dirRAWD when derived using a purely manual AAA segmentation compared with using semiautomated AAA segmentation (P = .55).

Conclusions

We found accurate and automated RAWD measurements were feasible with clinically insignificant errors. Using semiautomated AAA segmentations for deformable image registration methods did not alter maximal dirRAWD accuracy compared with using manual AAA segmentations. Future work will compare dirRAWD with finite element analysis–derived regional wall stress and determine whether dirRAWD might serve as an independent predictor of rupture risk.

Current abdominal aortic aneurysm (AAA) surveillance methods are limited to assessments of the maximal diameter, which cannot accurately predict for AAA expansion and rupture risk. Automated assessment of AAA expansion across the entire three-dimensional geometry of the aneurysm could better describe aneurysm growth and could substantially inform management decisions, including the indications for repair. We have developed an accurate and streamlined approach to assessing local three-dimensional AAA expansion with submillimeter accuracy using computed tomography imaging obtained during routine aneurysm surveillance. This novel process does not require significant user expertise nor computer processing power and can be performed using open-source software readily accessible to both scientists and clinicians.

Parameterization, geometric modeling, and isogeometric analysis of tricuspid valves

Approximately 1.6 million patients in the United States are affected by tricuspid valve regurgitation, which occurs when the tricuspid valve does not close properly to prevent backward blood flow into the right atrium. Despite its critical role in proper cardiac function, the tricuspid valve has received limited research attention compared to the mitral and aortic valves on the left side of the heart. As a result, proper valvular function and the pathologies that may cause dysfunction remain poorly understood. To promote further investigations of the biomechanical behavior and response of the tricuspid valve, this work establishes a parameter-based approach that provides a template for tricuspid valve modeling and simulation. The proposed tricuspid valve parameterization presents a comprehensive description of the leaflets and the complex chordae tendineae for capturing the typical three-cusp structural deformation observed from medical data. This simulation framework develops a practical procedure for modeling tricuspid valves and offers a robust, flexible approach to analyze the performance and effectiveness of various valve configurations using isogeometric analysis. The proposed methods also establish a baseline to examine the tricuspid valve's structural deformation, perform future investigations of native valve configurations under healthy and disease conditions, and optimize prosthetic valve designs.

Fitting unbranching skeletal structures to objects

Representing an object by a skeletal structure can be powerful for statistical shape analysis if there is good correspondence of the representations within a population. Many anatomic objects have a genus-zero boundary and can be represented by a smooth unbranching skeletal structure that can be discretely approximated. We describe how to compute such a discrete skeletal structure ("d-s-rep") for an individual 3D shape with the desired correspondence across cases. The method involves fitting a d-s-rep to an input representation of an object's boundary. A good fit is taken to be one whose skeletally implied boundary well approximates the target surface in terms of low order geometric boundary properties: (1) positions, (2) tangent fields, (3) various curvatures. Our method involves a two-stage framework that first, roughly yet consistently fits a skeletal structure to each object and second, refines the skeletal structure such that the shape of the implied boundary well approximates that of the object. The first stage uses a stratified diffeomorphism to produce topologically non-self-overlapping, smooth and unbranching skeletal structures for each object of a population. The second stage uses loss terms that measure geometric disagreement between the skeletally implied boundary and the target boundary and avoid self-overlaps in the boundary. By minimizing the total loss, we end up with a good d-s-rep for each individual shape. We demonstrate such d-s-reps for various human brain structures. The framework is accessible and extensible by clinical users, researchers and developers as an extension of SlicerSALT, which is based on 3D Slicer.

In Vivo Image-Based 4D Modeling of Competent and Regurgitant Mitral Valve Dynamics

BACKGROUND

In vivo characterization of mitral valve dynamics relies on image analysis algorithms that accurately reconstruct valve morphology and motion from clinical images. The goal of such algorithms is to provide patient-specific descriptions of both competent and regurgitant mitral valves, which can be used as input to biomechanical analyses and provide insights into the pathophysiology of diseases like ischemic mitral regurgitation (IMR).

OBJECTIVE

The goal is to generate accurate image-based representations of valve dynamics that visually and quantitatively capture normal and pathological valve function.

METHODS

We present a novel framework for 4D segmentation and geometric modeling of the mitral valve in real-time 3D echocardiography (rt-3DE), an imaging modality used for pre-operative surgical planning of mitral interventions. The framework integrates groupwise multi-atlas label fusion and template-based medial modeling with Kalman filtering to generate quantitatively descriptive and temporally consistent models of valve dynamics.

RESULTS

The algorithm is evaluated on rt-3DE data series from 28 patients: 14 with normal mitral valve morphology and 14 with severe IMR. In these 28 data series that total 613 individual 3DE images, each 3D mitral valve segmentation is validated against manual tracing, and temporal consistency between segmentations is demonstrated.

CONCLUSIONS

Automated 4D image analysis allows for reliable non-invasive modeling of the mitral valve over the cardiac cycle for comparison of annular and leaflet dynamics in pathological and normal mitral valves. Future studies can apply this algorithm to cardiovascular mechanics applications, including patient-specific strain estimation, fluid dynamics simulation, inverse finite element analysis, and risk stratification for surgical treatment.

A patient-specific mass-spring model for biomechanical simulation of aortic root tissue during transcatheter aortic valve implantation

The success of transcatheter aortic valve implantation (TAVI) is highly dependent on the prediction of the interaction between the prosthesis and the aortic root anatomy. The simulation of the surgical procedure may be useful to guide artificial valve selection and delivery, nevertheless the introduction of simulation models into the clinical workflow is often hindered by model complexity and computational burden. To address this point, we introduced a patient-specific mass-spring model (MSM) with viscous damping, as a good trade-off between simulation accuracy and time-efficiency. The anatomical model consisted of a hexahedral mesh, segmented from pre-procedural patient-specific cardiac computer tomographic (CT) images of the aortic root, including valve leaflets and attached calcifications. Nodal forces were represented by linear-elastic springs acting on edges and angles. A fast integration approach based on the modulation of nodal masses was also tested. The model was validated on seven patients, comparing simulation results with post-procedural CT images with respect to calcification and aortic wall position. The validation showed that the MSM was able to predict calcification displacement with an average accuracy of 1.72 mm and 1.54 mm for the normal and fast integration approaches, respectively. Wall displacement root mean squared error after valve expansion was about 1 mm for both approaches, showing an improved matching with respect to the pre-procedural configuration. In terms of computational burden, the fast integration approach allowed a consistent reduction of the computational times, which decreased from 36 h to 21.8 min per 100 K hexahedra. Our findings suggest that the proposed linear-elastic MSM model may provide good accuracy and reduced computational times for TAVI simulations, fostering its inclusion in clinical routines.

Towards intelligent robust detection of anatomical structures in incomplete volumetric data

Robust and fast detection of anatomical structures represents an important component of medical image analysis technologies. Current solutions for anatomy detection are based on machine learning, and are generally driven by suboptimal and exhaustive search strategies. In particular, these techniques do not effectively address cases of incomplete data, i.e., scans acquired with a partial field-of-view. We address these challenges by following a new paradigm, which reformulates the detection task to teaching an intelligent artificial agent how to actively search for an anatomical structure. Using the principles of deep reinforcement learning with multi-scale image analysis, artificial agents are taught optimal navigation paths in the scale-space representation of an image, while accounting for structures that are missing from the field-of-view. The spatial coherence of the observed anatomical landmarks is ensured using elements from statistical shape modeling and robust estimation theory. Experiments show that our solution outperforms marginal space deep learning, a powerful deep learning method, at detecting different anatomical structures without any failure. The dataset contains 5043 3D-CT volumes from over 2000 patients, totaling over 2,500,000 image slices. In particular, our solution achieves 0% false-positive and 0% false-negative rates at detecting whether the landmarks are captured in the field-of-view of the scan (excluding all border cases), with an average detection accuracy of 2.78 mm. In terms of runtime, we reduce the detection-time of the marginal space deep learning method by 20-30 times to under 40 ms, an unmatched performance for high resolution incomplete 3D-CT data.

Multi-resolution geometric modeling of the mitral heart valve leaflets

An essential element of cardiac function, the mitral valve (MV) ensures proper directional blood flow between the left heart chambers. Over the past two decades, computational simulations have made marked advancements toward providing powerful predictive tools to better understand valvular function and improve treatments for MV disease. However, challenges remain in the development of robust means for the quantification and representation of MV leaflet geometry. In this study, we present a novel modeling pipeline to quantitatively characterize and represent MV leaflet surface geometry. Our methodology utilized a two-part additive decomposition of the MV geometric features to decouple the macro-level general leaflet shape descriptors from the leaflet fine-scale features. First, the general shapes of five ovine MV leaflets were modeled using superquadric surfaces. Second, the finer-scale geometric details were captured, quantified, and reconstructed via a 2D Fourier analysis with an additional sparsity constraint. This spectral approach allowed us to easily control the level of geometric details in the reconstructed geometry. The results revealed that our methodology provided a robust and accurate approach to develop MV-specific models with an adjustable level of spatial resolution and geometric detail. Such fully customizable models provide the necessary means to perform computational simulations of the MV at a range of geometric accuracies in order to identify the level of complexity required to achieve predictive MV simulations.

Image Segmentation and Modeling of the Pediatric Tricuspid Valve in Hypoplastic Left Heart Syndrome

Hypoplastic left heart syndrome (HLHS) is a single-ventricle congenital heart disease that is fatal if left unpalliated. In HLHS patients, the tricuspid valve is the only functioning atrioventricular valve, and its competence is therefore critical. This work demonstrates the first automated strategy for segmentation, modeling, and morphometry of the tricuspid valve in transthoracic 3D echocardiographic (3DE) images of pediatric patients with HLHS. After initial landmark placement, the automated segmentation step uses multi-atlas label fusion and the modeling approach uses deformable modeling with medial axis representation to produce patient-specific models of the tricuspid valve that can be comprehensively and quantitatively assessed. In a group of 16 pediatric patients, valve segmentation and modeling attains an accuracy (mean boundary displacement) of 0.8 ± 0.2 mm relative to manual tracing and shows consistency in annular and leaflet measurements. In the future, such image-based tools have the potential to improve understanding and evaluation of tricuspid valve morphology in HLHS and guide strategies for patient care.

Fully Automatic 3-D-TEE Segmentation for the Planning of Transcatheter Aortic Valve Implantation

A novel fully automatic framework for aortic valve (AV) trunk segmentation in three-dimensional (3-D) transesophageal echocardiography (TEE) datasets is proposed. The methodology combines a previously presented semiautomatic segmentation strategy by using shape-based B-spline Explicit Active Surfaces with two novel algorithms to automate the quantification of relevant AV measures. The first combines a fast rotation-invariant 3-D generalized Hough transform with a vessel-like dark tube detector to initialize the segmentation. After segmenting the AV wall, the second algorithm focuses on aligning this surface with the reference ones in order to estimate the short-axis (SAx) planes (at the left ventricular outflow tract, annulus, sinuses of Valsalva, and sinotubular junction) in which to perform the measurements. The framework has been tested in 20 3-D-TEE datasets with both stenotic and nonstenotic AVs. The initialization algorithm presented a median error of around 3 mm for the AV axis endpoints, with an overall feasibility of 90%. In its turn, the SAx detection algorithm showed to be highly reproducible, with indistinguishable results compared with the variability found between the experts' defined planes. Automatically extracted measures at the four levels showed a good agreement with the experts' ones, with limits of agreement similar to the interobserver variability. Moreover, a validation set of 20 additional stenotic AV datasets corroborated the method's applicability and accuracy. The proposed approach mitigates the variability associated with the manual quantification while significantly reducing the required analysis time (12 s versus 5 to 10 min), which shows its appeal for automatic dimensioning of the AV morphology in 3-D-TEE for the planning of transcatheter AV implantation.

In-vivo heterogeneous functional and residual strains in human aortic valve leaflets

Residual and physiological functional strains in soft tissues are known to play an important role in modulating organ stress distributions. Yet, no known comprehensive information on residual strains exist, or non-invasive techniques to quantify in-vivo deformations for the aortic valve (AV) leaflets. Herein we present a completely non-invasive approach for determining heterogeneous strains - both functional and residual - in semilunar valves and apply it to normal human AV leaflets. Transesophageal 3D echocardiographic (3DE) images of the AV were acquired from open-heart transplant patients, with each AV leaflet excised after heart explant and then imaged in a flattened configuration ex-vivo. Using an established spline parameterization of both 3DE segmentations and digitized ex-vivo images (Aggarwal et al., 2014), surface strains were calculated for deformation between the ex-vivo and three in-vivo configurations: fully open, just-coapted, and fully-loaded. Results indicated that leaflet area increased by an average of 20% from the ex-vivo to in-vivo open states, with a highly heterogeneous strain field. The increase in area from open to just-coapted state was the highest at an average of 25%, while that from just-coapted to fully-loaded remained almost unaltered. Going from the ex-vivo to in-vivo mid-systole configurations, the leaflet area near the basal attachment shrank slightly, whereas the free edge expanded by ~10%. This was accompanied by a 10° -20° shear along the circumferential-radial direction. Moreover, the principal stretches aligned approximately with the circumferential and radial directions for all cases, with the highest stretch being along the radial direction. Collectively, these results indicated that even though the AV did not support any measurable pressure gradient in the just-coapted state, the leaflets were significantly pre-strained with respect to the excised state. Furthermore, the collagen fibers of the leaflet were almost fully recruited in the just-coapted state, making the leaflet very stiff with marginal deformation under full pressure. Lastly, the deformation was always higher in the radial direction and lower along the circumferential one, the latter direction made stiffer by the preferential alignment of collagen fibers. These results provide significant insight into the distribution of residual strains and the in-vivo strains encountered during valve opening and closing in AV leaflets, and will form an important component of the tool that can evaluate valve׳s functional properties in a non-invasive manner.

Marginal Space Deep Learning: Efficient Architecture for Volumetric Image Parsing

Robust and fast solutions for anatomical object detection and segmentation support the entire clinical workflow from diagnosis, patient stratification, therapy planning, intervention and follow-up. Current state-of-the-art techniques for parsing volumetric medical image data are typically based on machine learning methods that exploit large annotated image databases. Two main challenges need to be addressed, these are the efficiency in scanning high-dimensional parametric spaces and the need for representative image features which require significant efforts of manual engineering. We propose a pipeline for object detection and segmentation in the context of volumetric image parsing, solving a two-step learning problem: anatomical pose estimation and boundary delineation. For this task we introduce Marginal Space Deep Learning (MSDL), a novel framework exploiting both the strengths of efficient object parametrization in hierarchical marginal spaces and the automated feature design of Deep Learning (DL) network architectures. In the 3D context, the application of deep learning systems is limited by the very high complexity of the parametrization. More specifically 9 parameters are necessary to describe a restricted affine transformation in 3D, resulting in a prohibitive amount of billions of scanning hypotheses. The mechanism of marginal space learning provides excellent run-time performance by learning classifiers in clustered, high-probability regions in spaces of gradually increasing dimensionality. To further increase computational efficiency and robustness, in our system we learn sparse adaptive data sampling patterns that automatically capture the structure of the input. Given the object localization, we propose a DL-based active shape model to estimate the non-rigid object boundary. Experimental results are presented on the aortic valve in ultrasound using an extensive dataset of 2891 volumes from 869 patients, showing significant improvements of up to 45.2% over the state-of-the-art. To our knowledge, this is the first successful demonstration of the DL potential to detection and segmentation in full 3D data with parametrized representations.