Fully automatic segmentation of the mitral leaflets in 3D transesophageal echocardiographic images using multi-atlas joint label fusion and deformable medial modeling

Automatic 4D mitral valve segmentation from transesophageal echocardiography: a semi-supervised learning approach

Performing automatic and standardized 4D TEE segmentation and mitral valve analysis is challenging due to the limitations of echocardiography and the scarcity of manually annotated 4D images. This work proposes a semi-supervised training strategy using pseudo labelling for MV segmentation in 4D TEE; it employs a Teacher-Student framework to ensure reliable pseudo-label generation. 120 4D TEE recordings from 60 candidates for MV repair are used. The Teacher model, an ensemble of three convolutional neural networks, is trained on end-systole and end-diastole frames and is used to generate MV pseudo-segmentations on intermediate frames of the cardiac cycle. The pseudo-annotated frames augment the Student model's training set, improving segmentation accuracy and temporal consistency. The Student outperforms individual Teachers, achieving a Dice score of 0.82, an average surface distance of 0.37 mm, and a 95% Hausdorff distance of 1.72 mm for MV leaflets. The Student model demonstrates reliable frame-by-frame MV segmentation, accurately capturing leaflet morphology and dynamics throughout the cardiac cycle, with a significant reduction in inference time compared to the ensemble. This approach greatly reduces manual annotation workload and ensures reliable, repeatable, and time-efficient MV analysis. Our method holds strong potential to enhance the precision and efficiency of MV diagnostics and treatment planning in clinical settings.

Euclidean and Shape-Based Analysis of the Dynamic Mitral Annulus in Children using a Novel Open-Source Framework

BACKGROUND

The dynamic shape of the normal adult mitral annulus has been shown to be important to mitral valve function. However, annular dynamics of the healthy mitral valve in children have yet to be explored. The aim of this study was to model and quantify the shape and major modes of variation of pediatric mitral valve annuli in four phases of the cardiac cycle using transthoracic echocardiography.

METHODS

The mitral valve annuli of 100 children and young adults with normal findings on three-dimensional echocardiography were modeled in four different cardiac phases using the SlicerHeart extension for 3D Slicer. Annular metrics were quantified using SlicerHeart, and optimal normalization to body surface area was explored. Mean annular shapes and the principal components of variation were computed using custom code implemented in a new SlicerHeart module (Annulus Shape Analyzer). Shape was regressed over metrics of age and body surface area, and mean shapes for five age-stratified groups were generated.

RESULTS

The ratio of annular height to commissural width of the mitral valve ("saddle shape") changed significantly throughout age for systolic phases (P < .001) but within a narrow range (median range, 0.20-0.25). Annular metrics changed statistically significantly between the diastolic and systolic phases of the cardiac cycle. Visually, the annular shape was maintained with respect to age and body surface area. Principal-component analysis revealed that the pediatric mitral annulus varies primarily in size (mode 1), ratio of annular height to commissural width (mode 2), and sphericity (mode 3).

CONCLUSIONS

The saddle-shaped mitral annulus is maintained throughout childhood but varies significantly throughout the cardiac cycle. The major modes of variation in the pediatric mitral annulus are due to size, ratio of annular height to commissural width, and sphericity. The generation of age- and size-specific mitral annular shapes may inform the development of appropriately scaled absorbable or expandable mitral annuloplasty rings for children.

Automatic landmark identification in cone-beam computed tomography

OBJECTIVE

To present and validate an open-source fully automated landmark placement (ALICBCT) tool for cone-beam computed tomography scans.

MATERIALS AND METHODS

One hundred and forty-three large and medium field of view cone-beam computed tomography (CBCT) were used to train and test a novel approach, called ALICBCT that reformulates landmark detection as a classification problem through a virtual agent placed inside volumetric images. The landmark agents were trained to navigate in a multi-scale volumetric space to reach the estimated landmark position. The agent movements decision relies on a combination of DenseNet feature network and fully connected layers. For each CBCT, 32 ground truth landmark positions were identified by 2 clinician experts. After validation of the 32 landmarks, new models were trained to identify a total of 119 landmarks that are commonly used in clinical studies for the quantification of changes in bone morphology and tooth position.

RESULTS

Our method achieved a high accuracy with an average of 1.54 ± 0.87 mm error for the 32 landmark positions with rare failures, taking an average of 4.2 second computation time to identify each landmark in one large 3D-CBCT scan using a conventional GPU.

CONCLUSION

The ALICBCT algorithm is a robust automatic identification tool that has been deployed for clinical and research use as an extension in the 3D Slicer platform allowing continuous updates for increased precision.

Automated estimation of mitral annular plane systolic excursion by artificial intelligence from 3D ultrasound recordings

Perioperative monitoring of cardiac function is beneficial for early detection of cardiovascular complications. The standard of care for cardiac monitoring performed by trained cardiologists and anesthesiologists involves a manual and qualitative evaluation of ultrasound imaging, which is a time-demanding and resource-intensive process with intraobserver- and interobserver variability. In practice, such measures can only be performed a limited number of times during the intervention. To overcome these difficulties, this study presents a robust method for automatic and quantitative monitoring of cardiac function based on 3D transesophageal echocardiography (TEE) B-mode ultrasound recordings of the left ventricle (LV). Such an assessment obtains consistent measurements and can produce a near real-time evaluation of ultrasound imagery. Hence, the presented method is time-saving and results in increased accessibility. The mitral annular plane systolic excursion (MAPSE), characterizing global LV function, is estimated by landmark detection and cardiac view classification of two-dimensional images extracted along the long-axis of the ultrasound volume. MAPSE estimation directly from 3D TEE recordings is beneficial since it removes the need for manual acquisition of cardiac views, hence decreasing the need for interference by physicians. Two convolutional neural networks (CNNs) were trained and tested on acquired ultrasound data of 107 patients, and MAPSE estimates were compared to clinically obtained references in a blinded study including 31 patients. The proposed method for automatic MAPSE estimation had low bias and low variability in comparison to clinical reference measures. The method accomplished a mean difference for MAPSE estimates of (-0.16±1.06) mm. Thus, the results did not show significant systematic errors. The obtained bias and variance of the method were comparable to inter-observer variability of clinically obtained MAPSE measures on 2D TTE echocardiography. The novel pipeline proposed in this study has the potential to enhance cardiac monitoring in perioperative- and intensive care settings.

Modeling of the Tricuspid Valve and Right Ventricle in Hypoplastic Left Heart Syndrome With a Fontan Circulation

BACKGROUND

In hypoplastic left heart syndrome, tricuspid regurgitation (TR) is associated with circulatory failure and death. We hypothesized that the tricuspid valve (TV) structure of patients with hypoplastic left heart syndrome with a Fontan circulation and moderate or greater TR differs from those with mild or less TR, and that right ventricle volume is associated with TV structure and dysfunction.

METHODS

TV of 100 patients with hypoplastic left heart syndrome and a Fontan circulation were modeled using transthoracic 3-dimensional echocardiograms and custom software in SlicerHeart. Associations of TV structure to TR grade and right ventricle function and volume were investigated. Shape parameterization and analysis was used to calculate the mean shape of the TV leaflets, their principal modes of variation, and to characterize associations of TV leaflet shape to TR.

RESULTS

In univariate modeling, patients with moderate or greater TR had larger TV annular diameters and area, greater annular distance between the anteroseptal commissure and anteroposterior commissure, greater leaflet billow volume, and more laterally directed anterior papillary muscle angles compared to valves with mild or less TR (all P<0.001). In multivariate modeling greater total billow volume, lower anterior papillary muscle angle, and greater distance between the anteroposterior commissure and anteroseptal commissure were associated with moderate or greater TR (P<0.001, C statistic=0.85). Larger right ventricle volumes were associated with moderate or greater TR (P<0.001). TV shape analysis revealed structural features associated with TR, but also highly heterogeneous TV leaflet structure.

CONCLUSIONS

Moderate or greater TR in patients with hypoplastic left heart syndrome with a Fontan circulation is associated with greater leaflet billow volume, a more laterally directed anterior papillary muscle angle, and greater annular distance between the anteroseptal commissure and anteroposterior commissure. However, there is significant heterogeneity of structure in the TV leaflets in regurgitant valves. Given this variability, an image-informed patient-specific approach to surgical planning may be needed to achieve optimal outcomes in this vulnerable and challenging population.

SlicerHeart: An open-source computing platform for cardiac image analysis and modeling

Cardiovascular disease is a significant cause of morbidity and mortality in the developed world. 3D imaging of the heart's structure is critical to the understanding and treatment of cardiovascular disease. However, open-source tools for image analysis of cardiac images, particularly 3D echocardiographic (3DE) data, are limited. We describe the rationale, development, implementation, and application of SlicerHeart, a cardiac-focused toolkit for image analysis built upon 3D Slicer, an open-source image computing platform. We designed and implemented multiple Python scripted modules within 3D Slicer to import, register, and view 3DE data, including new code to volume render and crop 3DE. In addition, we developed dedicated workflows for the modeling and quantitative analysis of multi-modality image-derived heart models, including heart valves. Finally, we created and integrated new functionality to facilitate the planning of cardiac interventions and surgery. We demonstrate application of SlicerHeart to a diverse range of cardiovascular modeling and simulation including volume rendering of 3DE images, mitral valve modeling, transcatheter device modeling, and planning of complex surgical intervention such as cardiac baffle creation. SlicerHeart is an evolving open-source image processing platform based on 3D Slicer initiated to support the investigation and treatment of congenital heart disease. The technology in SlicerHeart provides a robust foundation for 3D image-based investigation in cardiovascular medicine.

Interplay between Artificial Intelligence and Biomechanics Modeling in the Cardiovascular Disease Prediction

Cardiovascular disease (CVD) is the most common cause of morbidity and mortality worldwide, and early accurate diagnosis is the key point for improving and optimizing the prognosis of CVD. Recent progress in artificial intelligence (AI), especially machine learning (ML) technology, makes it possible to predict CVD. In this review, we first briefly introduced the overview development of artificial intelligence. Then we summarized some ML applications in cardiovascular diseases, including ML-based models to directly predict CVD based on risk factors or medical imaging findings and the ML-based hemodynamics with vascular geometries, equations, and methods for indirect assessment of CVD. We also discussed case studies where ML could be used as the surrogate for computational fluid dynamics in data-driven models and physics-driven models. ML models could be a surrogate for computational fluid dynamics, accelerate the process of disease prediction, and reduce manual intervention. Lastly, we briefly summarized the research difficulties and prospected the future development of AI technology in cardiovascular diseases.

Fully Automated 3D Segmentation and Diffeomorphic Medial Modeling of the Left Ventricle Mitral Valve Complex in Ischemic Mitral Regurgitation

There is an urgent unmet need to develop a fully-automated image-based left ventricle mitral valve analysis tool to support surgical decision making for ischemic mitral regurgitation patients. This requires an automated tool for segmentation and modeling of the left ventricle and mitral valve from immediate pre-operative 3D transesophageal echocardiography. Previous works have presented methods for semi-automatically segmenting and modeling the mitral valve, but do not include the left ventricle and do not avoid self-intersection of the mitral valve leaflets during shape modeling. In this study, we develop and validate a fully automated algorithm for segmentation and shape modeling of the left ventricular mitral valve complex from pre-operative 3D transesophageal echocardiography. We performed a 3-fold nested cross validation study on two datasets from separate institutions to evaluate automated segmentations generated by nnU-net with the expert manual segmentation which yielded average overall Dice scores of 0.82±0.03 (set A), 0.87±0.08 (set B) respectively. A deformable medial template was subsequently fitted to the segmentation to generate shape models. Comparison of shape models to the manual and automatically generated segmentations resulted in an average Dice score of 0.93-0.94 and 0.75-0.81 for the left ventricle and mitral valve, respectively. This is a substantial step towards automatically analyzing the left ventricle mitral valve complex in the operating room.

Deep Learning for Computational Hemodynamics: A Brief Review of Recent Advances

Computational fluid dynamics (CFD) modeling of blood flow plays an important role in better understanding various medical conditions, designing more effective drug delivery systems, and developing novel diagnostic methods and treatments. However, despite significant advances in computational technology and resources, the expensive computational cost of these simulations still hinders their transformation from a research interest to a clinical tool. This bottleneck is even more severe for image-based, patient-specific CFD simulations with realistic boundary conditions and complex computational domains, which make such simulations excessively expensive. To address this issue, deep learning approaches have been recently explored to accelerate computational hemodynamics simulations. In this study, we review recent efforts to integrate deep learning with CFD and discuss the applications of this approach in solving hemodynamics problems, such as blood flow behavior in aorta and cerebral arteries. We also discuss potential future directions in the field. In this review, we suggest that incorporating physiologic understandings and underlying fluid mechanics laws in deep learning models will soon lead to a paradigm shift in the development novel non-invasive computational medical decisions.

Segmentation of Tricuspid Valve Leaflets From Transthoracic 3D Echocardiograms of Children With Hypoplastic Left Heart Syndrome Using Deep Learning

Hypoplastic left heart syndrome (HLHS) is a severe congenital heart defect in which the right ventricle and associated tricuspid valve (TV) alone support the circulation. TV failure is thus associated with heart failure, and the outcome of TV valve repair are currently poor. 3D echocardiography (3DE) can generate high-quality images of the valve, but segmentation is necessary for precise modeling and quantification. There is currently no robust methodology for rapid TV segmentation, limiting the clinical application of these technologies to this challenging population. We utilized a Fully Convolutional Network (FCN) to segment tricuspid valves from transthoracic 3DE. We trained on 133 3DE image-segmentation pairs and validated on 28 images. We then assessed the effect of varying inputs to the FCN using Mean Boundary Distance (MBD) and Dice Similarity Coefficient (DSC). The FCN with the input of an annular curve achieved a median DSC of 0.86 [IQR: 0.81-0.88] and MBD of 0.35 [0.23-0.4] mm for the merged segmentation and an average DSC of 0.77 [0.73-0.81] and MBD of 0.6 [0.44-0.74] mm for individual TV leaflet segmentation. The addition of commissural landmarks improved individual leaflet segmentation accuracy to an MBD of 0.38 [0.3-0.46] mm. FCN-based segmentation of the tricuspid valve from transthoracic 3DE is feasible and accurate. The addition of an annular curve and commissural landmarks improved the quality of the segmentations with MBD and DSC within the range of human inter-user variability. Fast and accurate FCN-based segmentation of the tricuspid valve in HLHS may enable rapid modeling and quantification, which in the future may inform surgical planning. We are now working to deploy this network for public use.

In Vivo Image-Based 4D Modeling of Competent and Regurgitant Mitral Valve Dynamics

BACKGROUND

In vivo characterization of mitral valve dynamics relies on image analysis algorithms that accurately reconstruct valve morphology and motion from clinical images. The goal of such algorithms is to provide patient-specific descriptions of both competent and regurgitant mitral valves, which can be used as input to biomechanical analyses and provide insights into the pathophysiology of diseases like ischemic mitral regurgitation (IMR).

OBJECTIVE

The goal is to generate accurate image-based representations of valve dynamics that visually and quantitatively capture normal and pathological valve function.

METHODS

We present a novel framework for 4D segmentation and geometric modeling of the mitral valve in real-time 3D echocardiography (rt-3DE), an imaging modality used for pre-operative surgical planning of mitral interventions. The framework integrates groupwise multi-atlas label fusion and template-based medial modeling with Kalman filtering to generate quantitatively descriptive and temporally consistent models of valve dynamics.

RESULTS

The algorithm is evaluated on rt-3DE data series from 28 patients: 14 with normal mitral valve morphology and 14 with severe IMR. In these 28 data series that total 613 individual 3DE images, each 3D mitral valve segmentation is validated against manual tracing, and temporal consistency between segmentations is demonstrated.

CONCLUSIONS

Automated 4D image analysis allows for reliable non-invasive modeling of the mitral valve over the cardiac cycle for comparison of annular and leaflet dynamics in pathological and normal mitral valves. Future studies can apply this algorithm to cardiovascular mechanics applications, including patient-specific strain estimation, fluid dynamics simulation, inverse finite element analysis, and risk stratification for surgical treatment.

Unsupervised Mitral Valve Tracking for Disease Detection in Echocardiogram Videos

In this work, a novel algorithmic scheme is developed that processes echocardiogram videos, and tracks the movement of the mitral valve leaflets, and thereby estimates whether the movement is symptomatic of a healthy or diseased heart. This algorithm uses automatic Otsu's thresholding to find a closed boundary around the left atrium, with the basic presumption that it is situated in the bottom right corner of the apical 4 chamber view. A centroid is calculated, and protruding prongs are taken within a 40-degree cone above the centroid, where the mitral valve is located. Binary images are obtained from the videos where the mitral valve leaflets have different pixel values than the cavity of the left atrium. Thus, the points where the prongs touch the valve will show where the mitral valve leaflets are located. The standard deviation of these points is used to calculate closeness of the leaflets. The estimation of the valve movement across subsequent frames is used to determine if the movement is regular, or affected by heart disease. Tests conducted with numerous videos containing both healthy and diseased hearts attest to our method's efficacy, with a key novelty in being fully unsupervised and computationally efficient.

Intraoperative post-annuloplasty three-dimensional valve analysis does not predict recurrent ischemic mitral regurgitation

BACKGROUND

High ischemic mitral regurgitation (IMR) recurrence rates continue to plague IMR repair with undersized ring annuloplasty. We have previously shown that pre-repair three-dimensional echocardiography (3DE) analysis is highly predictive of IMR recurrence. The objective of this study was to determine the quantitative change in 3DE annular and leaflet tethering parameters immediately after repair and to determine if intraoperative post-repair 3DE parameters would be able to predict IMR recurrence 6 months after repair.

METHODS

Intraoperative pre- and post-repair transesophageal real-time 3DE was performed in 35 patients undergoing undersized ring annuloplasty for IMR. An advanced modeling algorhythm was used to assess 3D annular geometry and regional leaflet tethering. IMR recurrence (≥ grade 2) was assessed with transthoracic echocardiography 6 months after repair.

RESULTS

Annuloplasty significantly reduced septolateral diameter, commissural width, annular area, and tethering volume and significantly increased all segmental tethering angles (except A2). Intraoperative post-repair annular geometry and leaflet tethering did not differ significantly between patients with recurrent IMR (n = 9) and patients with non-recurrent IMR (n = 26). No intraoperative post-repair predictors of IMR recurrence could be identified.

CONCLUSIONS

Undersized ring annuloplasty changes mitral geometry acutely, exacerbates leaflet tethering, and generally fixes IMR acutely, but it does not always fix the delicate underlying chronic problem of continued left ventricular dilatation and remodeling. This may explain why pre-repair 3D valve geometry (which reflects chronic left ventricular remodeling) is highly predictive of recurrent IMR, whereas immediate post-repair 3D valve geometry (which does not completely reflect chronic left ventricular remodeling anymore) is not.

Artificial intelligence and automation in valvular heart diseases

Artificial intelligence (AI) is gradually changing every aspect of social life, and healthcare is no exception. The clinical procedures that were supposed to, and could previously only be handled by human experts can now be carried out by machines in a more accurate and efficient way. The coming era of big data and the advent of supercomputers provides great opportunities to the development of AI technology for the enhancement of diagnosis and clinical decision-making. This review provides an introduction to AI and highlights its applications in the clinical flow of diagnosing and treating valvular heart diseases (VHDs). More specifically, this review first introduces some key concepts and subareas in AI. Secondly, it discusses the application of AI in heart sound auscultation and medical image analysis for assistance in diagnosing VHDs. Thirdly, it introduces using AI algorithms to identify risk factors and predict mortality of cardiac surgery. This review also describes the state-of-the-art autonomous surgical robots and their roles in cardiac surgery and intervention.

Moving beyond two-dimensional screens to interactive three-dimensional visualization in congenital heart disease

Beginning with the discovery of X-rays to the development of three-dimensional (3D) imaging, improvements in acquisition, post-processing, and visualization have provided clinicians with detailed information for increasingly accurate medical diagnosis and clinical management. This paper highlights advances in imaging technologies for congenital heart disease (CHD), medical adoption, and future developments required to improve pre-procedural and intra-procedural guidance.

A feasibility study of deep learning for predicting hemodynamics of human thoracic aorta

Numerical analysis methods including finite element analysis (FEA), computational fluid dynamics (CFD), and fluid-structure interaction (FSI) analysis have been used to study the biomechanics of human tissues and organs, as well as tissue-medical device interactions, and treatment strategies. However, for patient-specific computational analysis, complex procedures are usually required to set-up the models, and long computing time is needed to perform the simulation, preventing fast feedback to clinicians in time-sensitive clinical applications. In this study, by using machine learning techniques, we developed deep neural networks (DNNs) to directly estimate the steady-state distributions of pressure and flow velocity inside the thoracic aorta. After training on hemodynamic data from CFD simulations, the DNNs take as input a shape of the aorta and directly output the hemodynamic distributions in one second. The trained DNNs are capable of predicting the velocity magnitude field with an average error of 1.9608% and the pressure field with an average error of 1.4269%. This study demonstrates the feasibility and great potential of using DNNs as a fast and accurate surrogate model for hemodynamic analysis of large blood vessels.

Combining position-based dynamics and gradient vector flow for 4D mitral valve segmentation in TEE sequences

PURPOSE

For planning and guidance of minimally invasive mitral valve repair procedures, 3D+t transesophageal echocardiography (TEE) sequences are acquired before and after the intervention. The valve is then visually and quantitatively assessed in selected phases. To enable a quantitative assessment of valve geometry and pathological properties in all heart phases, as well as the changes achieved through surgery, we aim to provide a new 4D segmentation method.

METHODS

We propose a tracking-based approach combining gradient vector flow (GVF) and position-based dynamics (PBD). An open-state surface model of the valve is propagated through time to the closed state, attracted by the GVF field of the leaflet area. The PBD method ensures topological consistency during deformation. For evaluation, one expert in cardiac surgery annotated the closed-state leaflets in 10 TEE sequences of patients with normal and abnormal mitral valves, and defined the corresponding open-state models.

RESULTS

The average point-to-surface distance between the manual annotations and the final tracked model was [Formula: see text]. Qualitatively, four cases were satisfactory, five passable and one unsatisfactory. Each sequence could be segmented in 2-6 min.

CONCLUSION

Our approach enables to segment the mitral valve in 4D TEE image data with normal and pathological valve closing behavior. With this method, in addition to the quantification of the remaining orifice area, shape and dimensions of the coaptation zone can be analyzed and considered for planning and surgical result assessment.

Semi-automated Image Segmentation of the Midsystolic Left Ventricular Mitral Valve Complex in Ischemic Mitral Regurgitation

Ischemic mitral regurgitation (IMR) is primarily a left ventricular disease in which the mitral valve is dysfunctional due to ventricular remodeling after myocardial infarction. Current automated methods have focused on analyzing the mitral valve and left ventricle independently. While these methods have allowed for valuable insights into mechanisms of IMR, they do not fully integrate pathological features of the left ventricle and mitral valve. Thus, there is an unmet need to develop an automated segmentation algorithm for the left ventricular mitral valve complex, in order to allow for a more comprehensive study of this disease. The objective of this study is to generate and evaluate segmentations of the left ventricular mitral valve complex in pre-operative 3D transesophageal echocardiography using multi-atlas label fusion. These patient-specific segmentations could enable future statistical shape analysis for clinical outcome prediction and surgical risk stratification. In this study, we demonstrate a preliminary segmentation pipeline that achieves an average Dice coefficient of 0.78 ± 0.06.

A deep learning approach to estimate stress distribution: a fast and accurate surrogate of finite-element analysis

Structural finite-element analysis (FEA) has been widely used to study the biomechanics of human tissues and organs, as well as tissue-medical device interactions, and treatment strategies. However, patient-specific FEA models usually require complex procedures to set up and long computing times to obtain final simulation results, preventing prompt feedback to clinicians in time-sensitive clinical applications. In this study, by using machine learning techniques, we developed a deep learning (DL) model to directly estimate the stress distributions of the aorta. The DL model was designed and trained to take the input of FEA and directly output the aortic wall stress distributions, bypassing the FEA calculation process. The trained DL model is capable of predicting the stress distributions with average errors of 0.492% and 0.891% in the Von Mises stress distribution and peak Von Mises stress, respectively. This study marks, to our knowledge, the first study that demonstrates the feasibility and great potential of using the DL technique as a fast and accurate surrogate of FEA for stress analysis.

Multi-Scale Deep Reinforcement Learning for Real-Time 3D-Landmark Detection in CT Scans

Robust and fast detection of anatomical structures is a prerequisite for both diagnostic and interventional medical image analysis. Current solutions for anatomy detection are typically based on machine learning techniques that exploit large annotated image databases in order to learn the appearance of the captured anatomy. These solutions are subject to several limitations, including the use of suboptimal feature engineering techniques and most importantly the use of computationally suboptimal search-schemes for anatomy detection. To address these issues, we propose a method that follows a new paradigm by reformulating the detection problem as a behavior learning task for an artificial agent. We couple the modeling of the anatomy appearance and the object search in a unified behavioral framework, using the capabilities of deep reinforcement learning and multi-scale image analysis. In other words, an artificial agent is trained not only to distinguish the target anatomical object from the rest of the body but also how to find the object by learning and following an optimal navigation path to the target object in the imaged volumetric space. We evaluated our approach on 1487 3D-CT volumes from 532 patients, totaling over 500,000 image slices and show that it significantly outperforms state-of-the-art solutions on detecting several anatomical structures with no failed cases from a clinical acceptance perspective, while also achieving a 20-30 percent higher detection accuracy. Most importantly, we improve the detection-speed of the reference methods by 2-3 orders of magnitude, achieving unmatched real-time performance on large 3D-CT scans.

A noninvasive method for the determination of in vivo mitral valve leaflet strains

Assessment of mitral valve (MV) function is important in many diagnostic, prognostic, and surgical planning applications for treatment of MV disease. Yet, to date, there are no accepted noninvasive methods for determination of MV leaflet deformation, which is a critical metric of MV function. In this study, we present a novel, completely noninvasive computational method to estimate MV leaflet in-plane strains from clinical-quality real-time three-dimensional echocardiography (rt-3DE) images. The images were first segmented to produce meshed medial-surface leaflet geometries of the open and closed states. To establish material point correspondence between the two states, an image-based morphing pipeline was implemented within a finite element (FE) modeling framework in which MV closure was simulated by pressurizing the open-state geometry, and local corrective loads were applied to enforce the actual MV closed shape. This resulted in a complete map of local systolic leaflet membrane strains, obtained from the final FE mesh configuration. To validate the method, we utilized an extant in vitro database of fiducially labeled MVs, imaged in conditions mimicking both the healthy and diseased states. Our method estimated local anisotropic in vivo strains with less than 10% error and proved to be robust to changes in boundary conditions similar to those observed in ischemic MV disease. Next, we applied our methodology to ovine MVs imaged in vivo with rt-3DE and compared our results to previously published findings of in vivo MV strains in the same type of animal as measured using surgically sutured fiducial marker arrays. In regions encompassed by fiducial markers, we found no significant differences in circumferential(P = 0.240) or radial (P = 0.808) strain estimates between the marker-based measurements and our novel noninvasive method. This method can thus be used for model validation as well as for studies of MV disease and repair.

Three-Dimensional Mitral Valve Morphology in Children and Young Adults With Marfan Syndrome

BACKGROUND

Mitral valve (MV) prolapse is common in children with Marfan syndrome (MFS) and is associated with varying degrees of mitral regurgitation (MR). However, the three-dimensional (3D) morphology of the MV in children with MFS and its relation to the degree of MR are not known. The goals of this study were to describe the 3D morphology of the MV in children with MFS and to compare it to that in normal children.

METHODS

Three-dimensional transthoracic echocardiography was performed in 27 patients (3-21 years of age) meeting the revised Ghent criteria for MFS and 27 normal children matched by age (±1 year). The 3D geometry of the MV apparatus in midsystole was measured, and its association with clinical and two-dimensional echocardiographic parameters was examined.

RESULTS

Compared with age-matched control subjects, children with MFS had larger 3D annular areas (P < .02), smaller annular height/commissural width ratios (P < .001), greater billow volumes (P < .001), and smaller tenting heights, areas, and volumes (P < .001 for all). In multivariate modeling, larger leaflet billow volume in MFS was strongly associated with moderate or greater MR (P < .01). Intra- and interuser variability of 3D metrics was acceptable.

CONCLUSIONS

Children with MFS have flatter and more dilated MV annuli, greater billow volumes, and smaller tenting heights compared with normal control subjects. Larger billow volume is associated with MR. Three-dimensional MV quantification may contribute to the identification of patients with MFS and other connective tissue disorders. Further study of 3D MV geometry and its relation to the clinical progression of MV disease is warranted in this vulnerable population.

Selection Strategies for Atlas-Based Mosaicing of Left Atrial 3-D Transesophageal Echocardiography Data

Three-dimensional transesophageal echocardiography (TEE) provides real-time soft tissue information, but its use is hampered by its limited field of view. The mosaicing of multiple TEE views makes it possible to visualize a large structure, like the left atrium, in a single volume. To this end, an automatic registration method is required. Similarly to atlas-based segmentation approaches, atlas-based mosaicing (ABM) uses a full volume atlas set to moderate the onerous registration of the individual TEE views. The performance of ABM depends both on the quality of the involved registrations and on the selection of the optimal transformation from the candidate transformations that result from the various atlases. The study described here explored the performance of different selection strategies on multiview TEE data of the left atrium. We found that by incorporating two stages of transformation selection, using the image similarity and the conformity between the candidate transformations as selection criteria, the average registration error dropped below 3 mm with respect to manual registration of these data. Finally, we used this method for the automatic construction of a wide-view TEE volume of the left atrium.

Towards intelligent robust detection of anatomical structures in incomplete volumetric data

Robust and fast detection of anatomical structures represents an important component of medical image analysis technologies. Current solutions for anatomy detection are based on machine learning, and are generally driven by suboptimal and exhaustive search strategies. In particular, these techniques do not effectively address cases of incomplete data, i.e., scans acquired with a partial field-of-view. We address these challenges by following a new paradigm, which reformulates the detection task to teaching an intelligent artificial agent how to actively search for an anatomical structure. Using the principles of deep reinforcement learning with multi-scale image analysis, artificial agents are taught optimal navigation paths in the scale-space representation of an image, while accounting for structures that are missing from the field-of-view. The spatial coherence of the observed anatomical landmarks is ensured using elements from statistical shape modeling and robust estimation theory. Experiments show that our solution outperforms marginal space deep learning, a powerful deep learning method, at detecting different anatomical structures without any failure. The dataset contains 5043 3D-CT volumes from over 2000 patients, totaling over 2,500,000 image slices. In particular, our solution achieves 0% false-positive and 0% false-negative rates at detecting whether the landmarks are captured in the field-of-view of the scan (excluding all border cases), with an average detection accuracy of 2.78 mm. In terms of runtime, we reduce the detection-time of the marginal space deep learning method by 20-30 times to under 40 ms, an unmatched performance for high resolution incomplete 3D-CT data.

Signal dropout correction-based ultrasound segmentation for diastolic mitral valve modeling

Three-dimensional ultrasound segmentation of mitral valve (MV) at diastole is helpful for duplicating geometry and pathology in a patient-specific dynamic phantom. The major challenge is the signal dropout at leaflet regions in transesophageal echocardiography image data. Conventional segmentation approaches suffer from missing sonographic data leading to inaccurate MV modeling at leaflet regions. This paper proposes a signal dropout correction-based ultrasound segmentation method for diastolic MV modeling. The proposed method combines signal dropout correction, image fusion, continuous max-flow segmentation, and active contour segmentation techniques. The signal dropout correction approach is developed to recover the missing segmentation information. Once the signal dropout regions of TEE image data are recovered, the MV model can be accurately duplicated. Compared with other methods in current literature, the proposed algorithm exhibits lower computational cost. The experimental results show that the proposed algorithm gives competitive results for diastolic MV modeling compared with conventional segmentation algorithms, evaluated in terms of accuracy and efficiency.

Comparison of 3D Echocardiogram-Derived 3D Printed Valve Models to Molded Models for Simulated Repair of Pediatric Atrioventricular Valves

Mastering the technical skills required to perform pediatric cardiac valve surgery is challenging in part due to limited opportunity for practice. Transformation of 3D echocardiographic (echo) images of congenitally abnormal heart valves to realistic physical models could allow patient-specific simulation of surgical valve repair. We compared materials, processes, and costs for 3D printing and molding of patient-specific models for visualization and surgical simulation of congenitally abnormal heart valves. Pediatric atrioventricular valves (mitral, tricuspid, and common atrioventricular valve) were modeled from transthoracic 3D echo images using semi-automated methods implemented as custom modules in 3D Slicer. Valve models were then both 3D printed in soft materials and molded in silicone using 3D printed "negative" molds. Using pre-defined assessment criteria, valve models were evaluated by congenital cardiac surgeons to determine suitability for simulation. Surgeon assessment indicated that the molded valves had superior material properties for the purposes of simulation compared to directly printed valves (p < 0.01). Patient-specific, 3D echo-derived molded valves are a step toward realistic simulation of complex valve repairs but require more time and labor to create than directly printed models. Patient-specific simulation of valve repair in children using such models may be useful for surgical training and simulation of complex congenital cases.

From 4D Medical Images (CT, MRI, and Ultrasound) to 4D Structured Mesh Models of the Left Ventricular Endocardium for Patient-Specific Simulations

With cardiovascular disease (CVD) remaining the primary cause of death worldwide, early detection of CVDs becomes essential. The intracardiac flow is an important component of ventricular function, motion kinetics, wash-out of ventricular chambers, and ventricular energetics. Coupling between Computational Fluid Dynamics (CFD) simulations and medical images can play a fundamental role in terms of patient-specific diagnostic tools. From a technical perspective, CFD simulations with moving boundaries could easily lead to negative volumes errors and the sudden failure of the simulation. The generation of high-quality 4D meshes (3D in space + time) with 1-to-1 vertex becomes essential to perform a CFD simulation with moving boundaries. In this context, we developed a semiautomatic morphing tool able to create 4D high-quality structured meshes starting from a segmented 4D dataset. To prove the versatility and efficiency, the method was tested on three different 4D datasets (Ultrasound, MRI, and CT) by evaluating the quality and accuracy of the resulting 4D meshes. Furthermore, an estimation of some physiological quantities is accomplished for the 4D CT reconstruction. Future research will aim at extending the region of interest, further automation of the meshing algorithm, and generating structured hexahedral mesh models both for the blood and myocardial volume.

Spatiotemporal Segmentation and Modeling of the Mitral Valve in Real-Time 3D Echocardiographic Images

Transesophageal echocardiography is the primary imaging modality for preoperative assessment of mitral valves with ischemic mitral regurgitation (IMR). While there are well known echocardiographic insights into the 3D morphology of mitral valves with IMR, such as annular dilation and leaflet tethering, less is understood about how quantification of valve dynamics can inform surgical treatment of IMR or predict short-term recurrence of the disease. As a step towards filling this knowledge gap, we present a novel framework for 4D segmentation and geometric modeling of the mitral valve in real-time 3D echocardiography (rt-3DE). The framework integrates multi-atlas label fusion and template-based medial modeling to generate quantitatively descriptive models of valve dynamics. The novelty of this work is that temporal consistency in the rt-3DE segmentations is enforced during both the segmentation and modeling stages with the use of groupwise label fusion and Kalman filtering. The algorithm is evaluated on rt-3DE data series from 10 patients: five with normal mitral valve morphology and five with severe IMR. In these 10 data series that total 207 individual 3DE images, each 3DE segmentation is validated against manual tracing and temporal consistency between segmentations is demonstrated. The ultimate goal is to generate accurate and consistent representations of valve dynamics that can both visually and quantitatively provide insight into normal and pathological valve function.

Multi-resolution geometric modeling of the mitral heart valve leaflets

An essential element of cardiac function, the mitral valve (MV) ensures proper directional blood flow between the left heart chambers. Over the past two decades, computational simulations have made marked advancements toward providing powerful predictive tools to better understand valvular function and improve treatments for MV disease. However, challenges remain in the development of robust means for the quantification and representation of MV leaflet geometry. In this study, we present a novel modeling pipeline to quantitatively characterize and represent MV leaflet surface geometry. Our methodology utilized a two-part additive decomposition of the MV geometric features to decouple the macro-level general leaflet shape descriptors from the leaflet fine-scale features. First, the general shapes of five ovine MV leaflets were modeled using superquadric surfaces. Second, the finer-scale geometric details were captured, quantified, and reconstructed via a 2D Fourier analysis with an additional sparsity constraint. This spectral approach allowed us to easily control the level of geometric details in the reconstructed geometry. The results revealed that our methodology provided a robust and accurate approach to develop MV-specific models with an adjustable level of spatial resolution and geometric detail. Such fully customizable models provide the necessary means to perform computational simulations of the MV at a range of geometric accuracies in order to identify the level of complexity required to achieve predictive MV simulations.

Image Segmentation and Modeling of the Pediatric Tricuspid Valve in Hypoplastic Left Heart Syndrome

Hypoplastic left heart syndrome (HLHS) is a single-ventricle congenital heart disease that is fatal if left unpalliated. In HLHS patients, the tricuspid valve is the only functioning atrioventricular valve, and its competence is therefore critical. This work demonstrates the first automated strategy for segmentation, modeling, and morphometry of the tricuspid valve in transthoracic 3D echocardiographic (3DE) images of pediatric patients with HLHS. After initial landmark placement, the automated segmentation step uses multi-atlas label fusion and the modeling approach uses deformable modeling with medial axis representation to produce patient-specific models of the tricuspid valve that can be comprehensively and quantitatively assessed. In a group of 16 pediatric patients, valve segmentation and modeling attains an accuracy (mean boundary displacement) of 0.8 ± 0.2 mm relative to manual tracing and shows consistency in annular and leaflet measurements. In the future, such image-based tools have the potential to improve understanding and evaluation of tricuspid valve morphology in HLHS and guide strategies for patient care.

Atlas-Based Mosaicing of Left Atrial 3-D Transesophageal Echocardiography Images

Transesophageal echocardiography (TEE) is a promising imaging modality used to guide cardiac interventions, such as catheter ablation for the treatment of cardiac arrhythmias. These procedures rely on good visualization of the left atrium and pulmonary veins. To visualize these structures in a single volume, the acquisition, registration and fusion of multiple TEE views of the left atrium are required. We introduce atlas-based mosaicing as a method for the registration of images that are acquired according to a standardized protocol. Inspired by atlas-based segmentation approaches, compounded data of other patients serve as atlases for the registration of new data. The performance of atlas-based mosaicing is studied on 3-D TEE data of the left atrium and compared with that of regular pairwise registration. This study indicates that improved registration robustness and smaller registration errors are achieved with atlas-based mosaicing compared with regular pairwise registration. This is an important step toward the use of TEE for interventional guidance of ablation procedures.

Patient-specific pediatric silicone heart valve models based on 3D ultrasound

PURPOSE

Patient-specific heart and valve models have shown promise as training and planning tools for heart surgery, but physically realistic valve models remain elusive. Available proprietary, simulation-focused heart valve models are generic adult mitral valves and do not allow for patient-specific modeling as may be needed for rare diseases such as congenitally abnormal valves. We propose creating silicone valve models from a 3D-printed plastic mold as a solution that can be adapted to any individual patient and heart valve at a fraction of the cost of direct 3D-printing using soft materials.

METHODS

Leaflets of a pediatric mitral valve, a tricuspid valve in a patient with hypoplastic left heart syndrome, and a complete atrioventricular canal valve were segmented from ultrasound images. A custom software was developed to automatically generate molds for each valve based on the segmentation. These molds were 3D-printed and used to make silicone valve models. The models were designed with cylindrical rims of different sizes surrounding the leaflets, to show the outline of the valve and add rigidity. Pediatric cardiac surgeons practiced suturing on the models and evaluated them for use as surgical planning and training tools.

RESULTS

Five out of six surgeons reported that the valve models would be very useful as training tools for cardiac surgery. In this first iteration of valve models, leaflets were felt to be unrealistically thick or stiff compared to real pediatric leaflets. A thin tube rim was preferred for valve flexibility.

CONCLUSION

The valve models were well received and considered to be valuable and accessible tools for heart valve surgery training. Further improvements will be made based on surgeons' feedback.

Making three-dimensional echocardiography more tangible: a workflow for three-dimensional printing with echocardiographic data

Three-dimensional (3D) printing is a rapidly evolving technology with several potential applications in the diagnosis and management of cardiac disease. Recently, 3D printing (i.e. rapid prototyping) derived from 3D transesophageal echocardiography (TEE) has become possible. Due to the multiple steps involved and the specific equipment required for each step, it might be difficult to start implementing echocardiography-derived 3D printing in a clinical setting. In this review, we provide an overview of this process, including its logistics and organization of tools and materials, 3D TEE image acquisition strategies, data export, format conversion, segmentation, and printing. Generation of patient-specific models of cardiac anatomy from echocardiographic data is a feasible, practical application of 3D printing technology.

Patient-specific CFD models for intraventricular flow analysis from 3D ultrasound imaging: Comparison of three clinical cases

BACKGROUND

As the intracardiac flow field is affected by changes in shape and motility of the heart, intraventricular flow features can provide diagnostic indications. Ventricular flow patterns differ depending on the cardiac condition and the exploration of different clinical cases can provide insights into how flow fields alter in different pathologies.

METHODS

In this study, we applied a patient-specific computational fluid dynamics model of the left ventricle and mitral valve, with prescribed moving boundaries based on transesophageal ultrasound images for three cardiac pathologies, to verify the abnormal flow patterns in impaired hearts. One case (P1) had normal ejection fraction but low stroke volume and cardiac output, P2 showed low stroke volume and reduced ejection fraction, P3 had a dilated ventricle and reduced ejection fraction.

RESULTS

The shape of the ventricle and mitral valve, together with the pathology influence the flow field in the left ventricle, leading to distinct flow features. Of particular interest is the pattern of the vortex formation and evolution, influenced by the valvular orifice and the ventricular shape. The base-to-apex pressure difference of maximum 2mmHg is consistent with reported data.

CONCLUSION

We used a CFD model with prescribed boundary motion to describe the intraventricular flow field in three patients with impaired diastolic function. The calculated intraventricular flow dynamics are consistent with the diagnostic patient records and highlight the differences between the different cases. The integration of clinical images and computational techniques, therefore, allows for a deeper investigation intraventricular hemodynamics in patho-physiology.

Patient-specific CFD simulation of intraventricular haemodynamics based on 3D ultrasound imaging

BACKGROUND

The goal of this paper is to present a computational fluid dynamic (CFD) model with moving boundaries to study the intraventricular flows in a patient-specific framework. Starting from the segmentation of real-time transesophageal echocardiographic images, a CFD model including the complete left ventricle and the moving 3D mitral valve was realized. Their motion, known as a function of time from the segmented ultrasound images, was imposed as a boundary condition in an Arbitrary Lagrangian-Eulerian framework.

RESULTS

The model allowed for a realistic description of the displacement of the structures of interest and for an effective analysis of the intraventricular flows throughout the cardiac cycle. The model provides detailed intraventricular flow features, and highlights the importance of the 3D valve apparatus for the vortex dynamics and apical flow.

CONCLUSIONS

The proposed method could describe the haemodynamics of the left ventricle during the cardiac cycle. The methodology might therefore be of particular importance in patient treatment planning to assess the impact of mitral valve treatment on intraventricular flow dynamics.

Modeling the Myxomatous Mitral Valve With Three-Dimensional Echocardiography

BACKGROUND

Degenerative mitral valve disease is associated with variable and complex defects in valve morphology. Three-dimensional echocardiography (3DE) has shown promise in aiding preoperative planning for patients with this disease but to date has not been as transformative as initially predicted. The clinical usefulness of 3DE has been limited by the laborious methods currently required to extract quantitative data from the images.

METHODS

To maximize the utility of 3DE for preoperative valve evaluation, this work describes an automated 3DE image analysis method for generating models of the mitral valve that are well suited for both qualitative and quantitative assessment. The method is unique in that it captures detailed alterations in mitral leaflet and annular morphology and produces image-derived models with locally varying leaflet thickness. The method is evaluated on midsystolic transesophageal 3DE images acquired from 22 subjects with myxomatous degeneration and from 22 subjects with normal mitral valve morphology.

RESULTS

Relative to manual image analysis, the automated method accurately represents both normal and complex leaflet geometries with a mean boundary displacement error on the order of one image voxel. A detailed quantitative analysis of the valves is presented and reveals statistically significant differences between normal and myxomatous valves with respect to numerous aspects of annular and leaflet geometry.

CONCLUSIONS

This work demonstrates a successful methodology for the relatively rapid quantitative description of the complex mitral valve distortions associated with myxomatous degeneration. The methodology has the potential to significantly improve surgical planning for patients with complex mitral valve disease.

Marginal Space Deep Learning: Efficient Architecture for Volumetric Image Parsing

Robust and fast solutions for anatomical object detection and segmentation support the entire clinical workflow from diagnosis, patient stratification, therapy planning, intervention and follow-up. Current state-of-the-art techniques for parsing volumetric medical image data are typically based on machine learning methods that exploit large annotated image databases. Two main challenges need to be addressed, these are the efficiency in scanning high-dimensional parametric spaces and the need for representative image features which require significant efforts of manual engineering. We propose a pipeline for object detection and segmentation in the context of volumetric image parsing, solving a two-step learning problem: anatomical pose estimation and boundary delineation. For this task we introduce Marginal Space Deep Learning (MSDL), a novel framework exploiting both the strengths of efficient object parametrization in hierarchical marginal spaces and the automated feature design of Deep Learning (DL) network architectures. In the 3D context, the application of deep learning systems is limited by the very high complexity of the parametrization. More specifically 9 parameters are necessary to describe a restricted affine transformation in 3D, resulting in a prohibitive amount of billions of scanning hypotheses. The mechanism of marginal space learning provides excellent run-time performance by learning classifiers in clustered, high-probability regions in spaces of gradually increasing dimensionality. To further increase computational efficiency and robustness, in our system we learn sparse adaptive data sampling patterns that automatically capture the structure of the input. Given the object localization, we propose a DL-based active shape model to estimate the non-rigid object boundary. Experimental results are presented on the aortic valve in ultrasound using an extensive dataset of 2891 volumes from 869 patients, showing significant improvements of up to 45.2% over the state-of-the-art. To our knowledge, this is the first successful demonstration of the DL potential to detection and segmentation in full 3D data with parametrized representations.

Medially constrained deformable modeling for segmentation of branching medial structures: Application to aortic valve segmentation and morphometry

Deformable modeling with medial axis representation is a useful means of segmenting and parametrically describing the shape of anatomical structures in medical images. Continuous medial representation (cm-rep) is a "skeleton-first" approach to deformable medial modeling that explicitly parameterizes an object's medial axis and derives the object's boundary algorithmically. Although cm-rep has effectively been used to segment and model a number of anatomical structures with non-branching medial topologies, the framework is challenging to apply to objects with branching medial geometries since branch curves in the medial axis are difficult to parameterize. In this work, we demonstrate the first clinical application of a new "boundary-first" deformable medial modeling paradigm, wherein an object's boundary is explicitly described and constraints are imposed on boundary geometry to preserve the branching configuration of the medial axis during model deformation. This "boundary-first" framework is leveraged to segment and morphologically analyze the aortic valve apparatus in 3D echocardiographic images. Relative to manual tracing, segmentation with deformable medial modeling achieves a mean boundary error of 0.41 ± 0.10 mm (approximately one voxel) in 22 3DE images of normal aortic valves at systole. Deformable medial modeling is additionally demonstrated on pathological cases, including aortic stenosis, Marfan syndrome, and bicuspid aortic valve disease. This study demonstrates a promising approach for quantitative 3DE analysis of aortic valve morphology.