1
|
Automatic morphological classification of mitral valve diseases in echocardiographic images based on explainable deep learning methods. Int J Comput Assist Radiol Surg 2021; 17:413-425. [PMID: 34897594 DOI: 10.1007/s11548-021-02542-7] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2021] [Accepted: 11/30/2021] [Indexed: 10/19/2022]
Abstract
PURPOSE Carpentier's functional classification is a guide to explain the types of mitral valve regurgitation based on morphological features. There are four types of pathological morphologies, regardless of the presence or absence of mitral regurgitation: Type I, normal; Type II, mitral valve prolapse; Type IIIa, mitral valve stenosis; and Type IIIb, restricted mitral leaflet motion. The aim of this study was to automatically classify mitral valves using echocardiographic images. METHODS In our procedure, after the classification of apical 4-chamber (A4C) and parasternal long-axis (PLA) views, we extracted the systolic/diastolic phase of the cardiac cycle by calculating the left ventricular area. Six typical pre-trained models were fine-tuned with a 4-class model for the PLA and a 3-class model for the A4C views. As an additional contribution, to provide explainability, we applied the Gradient-weighted Class Activation Mapping (Grad-CAM) algorithm to visualize areas of echocardiographic images where the different models generated a prediction. RESULTS This approach conferred a proper understanding of where various networks "look" into echocardiographic images to predict the four types of pathological mitral valve morphologies. Considering the accuracy metric and Grad-CAM maps and by applying the Inception-ResNet-v2 architecture to classify Type II in the PLA view and ResNeXt50 architecture to classify the other three classes in the A4C view, we achieved an 80% rate of model accuracy in the test data set. CONCLUSIONS We suggest an explainable, fully automated, and rule-based procedure to classify the four types of mitral valve morphologies based on Carpentier's functional classification using deep learning on transthoracic echocardiographic images. Our study results infer the feasibility of the use of deep learning models to prepare quick and precise assessments of mitral valve morphologies in echocardiograms. According to our knowledge, our study is the first one that provides a public data set regarding the Carpentier classification of MV pathologies.
Collapse
|
2
|
Herz C, Pace DF, Nam HH, Lasso A, Dinh P, Flynn M, Cianciulli A, Golland P, Jolley MA. Segmentation of Tricuspid Valve Leaflets From Transthoracic 3D Echocardiograms of Children With Hypoplastic Left Heart Syndrome Using Deep Learning. Front Cardiovasc Med 2021; 8:735587. [PMID: 34957233 PMCID: PMC8696083 DOI: 10.3389/fcvm.2021.735587] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2021] [Accepted: 11/16/2021] [Indexed: 11/13/2022] Open
Abstract
Hypoplastic left heart syndrome (HLHS) is a severe congenital heart defect in which the right ventricle and associated tricuspid valve (TV) alone support the circulation. TV failure is thus associated with heart failure, and the outcome of TV valve repair are currently poor. 3D echocardiography (3DE) can generate high-quality images of the valve, but segmentation is necessary for precise modeling and quantification. There is currently no robust methodology for rapid TV segmentation, limiting the clinical application of these technologies to this challenging population. We utilized a Fully Convolutional Network (FCN) to segment tricuspid valves from transthoracic 3DE. We trained on 133 3DE image-segmentation pairs and validated on 28 images. We then assessed the effect of varying inputs to the FCN using Mean Boundary Distance (MBD) and Dice Similarity Coefficient (DSC). The FCN with the input of an annular curve achieved a median DSC of 0.86 [IQR: 0.81-0.88] and MBD of 0.35 [0.23-0.4] mm for the merged segmentation and an average DSC of 0.77 [0.73-0.81] and MBD of 0.6 [0.44-0.74] mm for individual TV leaflet segmentation. The addition of commissural landmarks improved individual leaflet segmentation accuracy to an MBD of 0.38 [0.3-0.46] mm. FCN-based segmentation of the tricuspid valve from transthoracic 3DE is feasible and accurate. The addition of an annular curve and commissural landmarks improved the quality of the segmentations with MBD and DSC within the range of human inter-user variability. Fast and accurate FCN-based segmentation of the tricuspid valve in HLHS may enable rapid modeling and quantification, which in the future may inform surgical planning. We are now working to deploy this network for public use.
Collapse
Affiliation(s)
- Christian Herz
- Children's Hospital of Philadelphia, Department of Anesthesia and Critical Care Medicine, Philadelphia, PA, United States
| | - Danielle F. Pace
- Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, United States
- Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School, Boston, MA, United States
| | - Hannah H. Nam
- Children's Hospital of Philadelphia, Department of Anesthesia and Critical Care Medicine, Philadelphia, PA, United States
| | - Andras Lasso
- Laboratory for Percutaneous Surgery, Queen's University, Kingston, ON, Canada
| | - Patrick Dinh
- Children's Hospital of Philadelphia, Department of Anesthesia and Critical Care Medicine, Philadelphia, PA, United States
| | - Maura Flynn
- Children's Hospital of Philadelphia, Department of Anesthesia and Critical Care Medicine, Philadelphia, PA, United States
| | - Alana Cianciulli
- Children's Hospital of Philadelphia, Department of Anesthesia and Critical Care Medicine, Philadelphia, PA, United States
| | - Polina Golland
- Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, United States
| | - Matthew A. Jolley
- Children's Hospital of Philadelphia, Department of Anesthesia and Critical Care Medicine, Philadelphia, PA, United States
- Division of Pediatric Cardiology, Children's Hospital of Philadelphia, Philadelphia, PA, United States
| |
Collapse
|
3
|
de Siqueira VS, Borges MM, Furtado RG, Dourado CN, da Costa RM. Artificial intelligence applied to support medical decisions for the automatic analysis of echocardiogram images: A systematic review. Artif Intell Med 2021; 120:102165. [PMID: 34629153 DOI: 10.1016/j.artmed.2021.102165] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2021] [Revised: 08/07/2021] [Accepted: 08/31/2021] [Indexed: 12/16/2022]
Abstract
The echocardiogram is a test that is widely used in Heart Disease Diagnoses. However, its analysis is largely dependent on the physician's experience. In this regard, artificial intelligence has become an essential technology to assist physicians. This study is a Systematic Literature Review (SLR) of primary state-of-the-art studies that used Artificial Intelligence (AI) techniques to automate echocardiogram analyses. Searches on the leading scientific article indexing platforms using a search string returned approximately 1400 articles. After applying the inclusion and exclusion criteria, 118 articles were selected to compose the detailed SLR. This SLR presents a thorough investigation of AI applied to support medical decisions for the main types of echocardiogram (Transthoracic, Transesophageal, Doppler, Stress, and Fetal). The article's data extraction indicated that the primary research interest of the studies comprised four groups: 1) Improvement of image quality; 2) identification of the cardiac window vision plane; 3) quantification and analysis of cardiac functions, and; 4) detection and classification of cardiac diseases. The articles were categorized and grouped to show the main contributions of the literature to each type of ECHO. The results indicate that the Deep Learning (DL) methods presented the best results for the detection and segmentation of the heart walls, right and left atrium and ventricles, and classification of heart diseases using images/videos obtained by echocardiography. The models that used Convolutional Neural Network (CNN) and its variations showed the best results for all groups. The evidence produced by the results presented in the tabulation of the studies indicates that the DL contributed significantly to advances in echocardiogram automated analysis processes. Although several solutions were presented regarding the automated analysis of ECHO, this area of research still has great potential for further studies to improve the accuracy of results already known in the literature.
Collapse
Affiliation(s)
- Vilson Soares de Siqueira
- Federal Institute of Tocantins, Av. Bernado Sayão, S/N, Santa Maria, Colinas do Tocantins, TO, Brazil; Federal University of Goias, Alameda Palmeiras, Quadra D, Câmpus Samambaia, Goiânia, GO, Brazil.
| | - Moisés Marcos Borges
- Diagnostic Imaging Center - CDI, Av. Portugal, 1155, St. Marista, Goiânia, GO, Brazil
| | - Rogério Gomes Furtado
- Diagnostic Imaging Center - CDI, Av. Portugal, 1155, St. Marista, Goiânia, GO, Brazil
| | - Colandy Nunes Dourado
- Diagnostic Imaging Center - CDI, Av. Portugal, 1155, St. Marista, Goiânia, GO, Brazil. http://www.cdigoias.com.br
| | - Ronaldo Martins da Costa
- Federal University of Goias, Alameda Palmeiras, Quadra D, Câmpus Samambaia, Goiânia, GO, Brazil.
| |
Collapse
|
4
|
Van Praet KM, Kempfert J, Jacobs S, Stamm C, Akansel S, Kofler M, Sündermann SH, Nazari Shafti TZ, Jakobs K, Holzendorf S, Unbehaun A, Falk V. Mitral valve surgery: current status and future prospects of the minimally invasive approach. Expert Rev Med Devices 2021; 18:245-260. [PMID: 33624569 DOI: 10.1080/17434440.2021.1894925] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
Abstract
Introduction: During the past five years the approach to procedural planning, operative techniques and perfusion strategies for minimally invasive mitral valve surgery (MIMVS) has evolved. With the goal to provide a maximum of patient safety the procedure has been modified according to individual patient characteristics and is largely based on preoperative imaging.Areas covered: In this review article we describe the important factors in image based therapy planning and simulation, different access strategies, the operative key-steps, a rationale use of devices, and highlight a few future developments in the field of MIMVS. Published studies were identified through pearl growing, citation chasing, a search of PubMed using the systematic review methods filter, and the authors' topic knowledge.Expert opinion: With the help of expert teams including surgeons specialized in mitral repair, anesthesiologists and perfusionists a broad spectrum of mitral valve pathologies and related pathologies can be treated with excellent functional outcomes. Avoiding procedure related complications is the key for success for any MIMVS program.
Collapse
Affiliation(s)
- Karel M Van Praet
- Department of Cardiothoracic and Vascular Surgery, German Heart Center Berlin, Berlin, Germany.,German Center for Cardiovascular Research (DZHK), Partner Site Berlin, Berlin, Germany
| | - Jörg Kempfert
- Department of Cardiothoracic and Vascular Surgery, German Heart Center Berlin, Berlin, Germany.,German Center for Cardiovascular Research (DZHK), Partner Site Berlin, Berlin, Germany
| | - Stephan Jacobs
- Department of Cardiothoracic and Vascular Surgery, German Heart Center Berlin, Berlin, Germany
| | - Christof Stamm
- Department of Cardiothoracic and Vascular Surgery, German Heart Center Berlin, Berlin, Germany
| | - Serdar Akansel
- Department of Cardiothoracic and Vascular Surgery, German Heart Center Berlin, Berlin, Germany
| | - Markus Kofler
- Department of Cardiothoracic and Vascular Surgery, German Heart Center Berlin, Berlin, Germany
| | - Simon H Sündermann
- Department of Cardiothoracic and Vascular Surgery, German Heart Center Berlin, Berlin, Germany.,German Center for Cardiovascular Research (DZHK), Partner Site Berlin, Berlin, Germany.,Department of Cardiothoracic Surgery, Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Timo Z Nazari Shafti
- Department of Cardiothoracic and Vascular Surgery, German Heart Center Berlin, Berlin, Germany.,German Center for Cardiovascular Research (DZHK), Partner Site Berlin, Berlin, Germany.,Berlin Institute of Health (BIH), Berlin, Germany
| | - Katharina Jakobs
- Institute for Anesthesiology, German Heart Center Berlin, Berlin, Germany
| | - Stefan Holzendorf
- Department of Perfusion, German Heart Center Berlin, Berlin, Germany
| | - Axel Unbehaun
- Department of Cardiothoracic and Vascular Surgery, German Heart Center Berlin, Berlin, Germany.,German Center for Cardiovascular Research (DZHK), Partner Site Berlin, Berlin, Germany
| | - Volkmar Falk
- Department of Cardiothoracic and Vascular Surgery, German Heart Center Berlin, Berlin, Germany.,German Center for Cardiovascular Research (DZHK), Partner Site Berlin, Berlin, Germany.,Department of Cardiothoracic Surgery, Charité - Universitätsmedizin Berlin, Berlin, Germany.,Berlin Institute of Health (BIH), Berlin, Germany.,Department of Health Sciences, ETH Zürich, Translational Cardiovascular Technologies, Switzerland
| |
Collapse
|
5
|
Chen X, Owen CA, Huang EC, Maggard BD, Latif RK, Clifford SP, Li J, Huang J. Artificial Intelligence in Echocardiography for Anesthesiologists. J Cardiothorac Vasc Anesth 2020; 35:251-261. [PMID: 32962932 DOI: 10.1053/j.jvca.2020.08.048] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/31/2020] [Accepted: 08/19/2020] [Indexed: 02/06/2023]
Abstract
Echocardiography is a unique diagnostic tool for intraoperative monitoring and assessment of patients with cardiovascular diseases. However, there are high levels of interoperator variations in echocardiography interpretations that could lead to inaccurate diagnosis and incorrect treatment. Furthermore, anesthesiologists are faced with the additional challenge to interpret echocardiography and make decisions in a limited timeframe from these complex data. The need for an automated, less operator-dependent process that enhances speed and accuracy of echocardiography analysis is crucial for anesthesiologists. Artificial intelligence is playing an increasingly important role in the medical field and could help anesthesiologists analyze complex echocardiographic data while adding increased accuracy and consistency to interpretation. This review aims to summarize practical use of artificial intelligence in echocardiography and discusses potential limitations and challenges in the future for anesthesiologists.
Collapse
Affiliation(s)
- Xia Chen
- Department of Anesthesiology, Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | | | | | - Brittany D Maggard
- Department of Anesthesiology and Perioperative Medicine, University of Louisville, Louisville, KY
| | - Rana K Latif
- Department of Anesthesiology and Perioperative Medicine, University of Louisville, Louisville, KY; Outcomes Research Consortium, Cleveland, Ohio, USA
| | - Sean P Clifford
- Department of Anesthesiology and Perioperative Medicine, University of Louisville, Louisville, KY
| | - Jinbao Li
- Department of Anesthesiology, Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Jiapeng Huang
- Department of Anesthesiology and Perioperative Medicine, University of Louisville, Louisville, KY; Department of Cardiovascular and Thoracic Surgery, University of Louisville, Louisville, KY.
| |
Collapse
|
6
|
Combining position-based dynamics and gradient vector flow for 4D mitral valve segmentation in TEE sequences. Int J Comput Assist Radiol Surg 2019; 15:119-128. [PMID: 31598891 DOI: 10.1007/s11548-019-02071-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2019] [Accepted: 09/30/2019] [Indexed: 10/25/2022]
Abstract
PURPOSE For planning and guidance of minimally invasive mitral valve repair procedures, 3D+t transesophageal echocardiography (TEE) sequences are acquired before and after the intervention. The valve is then visually and quantitatively assessed in selected phases. To enable a quantitative assessment of valve geometry and pathological properties in all heart phases, as well as the changes achieved through surgery, we aim to provide a new 4D segmentation method. METHODS We propose a tracking-based approach combining gradient vector flow (GVF) and position-based dynamics (PBD). An open-state surface model of the valve is propagated through time to the closed state, attracted by the GVF field of the leaflet area. The PBD method ensures topological consistency during deformation. For evaluation, one expert in cardiac surgery annotated the closed-state leaflets in 10 TEE sequences of patients with normal and abnormal mitral valves, and defined the corresponding open-state models. RESULTS The average point-to-surface distance between the manual annotations and the final tracked model was [Formula: see text]. Qualitatively, four cases were satisfactory, five passable and one unsatisfactory. Each sequence could be segmented in 2-6 min. CONCLUSION Our approach enables to segment the mitral valve in 4D TEE image data with normal and pathological valve closing behavior. With this method, in addition to the quantification of the remaining orifice area, shape and dimensions of the coaptation zone can be analyzed and considered for planning and surgical result assessment.
Collapse
|
7
|
Xia W, Moore J, Chen ECS, Xu Y, Ginty O, Bainbridge D, Peters TM. Signal dropout correction-based ultrasound segmentation for diastolic mitral valve modeling. J Med Imaging (Bellingham) 2018; 5:021214. [PMID: 29487886 PMCID: PMC5806032 DOI: 10.1117/1.jmi.5.2.021214] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2017] [Accepted: 01/04/2018] [Indexed: 11/14/2022] Open
Abstract
Three-dimensional ultrasound segmentation of mitral valve (MV) at diastole is helpful for duplicating geometry and pathology in a patient-specific dynamic phantom. The major challenge is the signal dropout at leaflet regions in transesophageal echocardiography image data. Conventional segmentation approaches suffer from missing sonographic data leading to inaccurate MV modeling at leaflet regions. This paper proposes a signal dropout correction-based ultrasound segmentation method for diastolic MV modeling. The proposed method combines signal dropout correction, image fusion, continuous max-flow segmentation, and active contour segmentation techniques. The signal dropout correction approach is developed to recover the missing segmentation information. Once the signal dropout regions of TEE image data are recovered, the MV model can be accurately duplicated. Compared with other methods in current literature, the proposed algorithm exhibits lower computational cost. The experimental results show that the proposed algorithm gives competitive results for diastolic MV modeling compared with conventional segmentation algorithms, evaluated in terms of accuracy and efficiency.
Collapse
Affiliation(s)
- Wenyao Xia
- Western University, Robarts Research Institute, Canada
- Western University, Department of Medical Biophysics, Canada
| | - John Moore
- Western University, Robarts Research Institute, Canada
| | - Elvis C. S. Chen
- Western University, Robarts Research Institute, Canada
- Western University, Department of Medical Biophysics, Canada
- Western University, Biomedical Engineering Graduate Program, Canada
| | - Yuanwei Xu
- Western University, Robarts Research Institute, Canada
| | - Olivia Ginty
- Western University, Robarts Research Institute, Canada
| | | | - Terry M. Peters
- Western University, Robarts Research Institute, Canada
- Western University, Department of Medical Biophysics, Canada
- Western University, Biomedical Engineering Graduate Program, Canada
| |
Collapse
|