1
|
Mathis M, Steffner KR, Subramanian H, Gill GP, Girardi NI, Bansal S, Bartels K, Khanna AK, Huang J. Overview and Clinical Applications of Artificial Intelligence and Machine Learning in Cardiac Anesthesiology. J Cardiothorac Vasc Anesth 2024; 38:1211-1220. [PMID: 38453558 PMCID: PMC10999327 DOI: 10.1053/j.jvca.2024.02.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/25/2023] [Revised: 01/30/2024] [Accepted: 02/05/2024] [Indexed: 03/09/2024]
Abstract
Artificial intelligence- (AI) and machine learning (ML)-based applications are becoming increasingly pervasive in the healthcare setting. This has in turn challenged clinicians, hospital administrators, and health policymakers to understand such technologies and develop frameworks for safe and sustained clinical implementation. Within cardiac anesthesiology, challenges and opportunities for AI/ML to support patient care are presented by the vast amounts of electronic health data, which are collected rapidly, interpreted, and acted upon within the periprocedural area. To address such challenges and opportunities, in this article, the authors review 3 recent applications relevant to cardiac anesthesiology, including depth of anesthesia monitoring, operating room resource optimization, and transthoracic/transesophageal echocardiography, as conceptual examples to explore strengths and limitations of AI/ML within healthcare, and characterize this evolving landscape. Through reviewing such applications, the authors introduce basic AI/ML concepts and methodologies, as well as practical considerations and ethical concerns for initiating and maintaining safe clinical implementation of AI/ML-based algorithms for cardiac anesthesia patient care.
Collapse
Affiliation(s)
- Michael Mathis
- Department of Anesthesiology, University of Michigan Medicine, Ann Arbor, MI
| | - Kirsten R Steffner
- Department of Anesthesiology, Perioperative and Pain Medicine, Stanford University School of Medicine, Stanford, CA
| | - Harikesh Subramanian
- Department of Anesthesiology and Perioperative Medicine, University of Pittsburgh, Pittsburgh, PA
| | - George P Gill
- Department of Anesthesiology, Cedars Sinai, Los Angeles, CA
| | | | - Sagar Bansal
- Department of Anesthesiology and Perioperative Medicine, University of Missouri School of Medicine, Columbia, MO
| | - Karsten Bartels
- Department of Anesthesiology, University of Nebraska Medical Center, Omaha, NE
| | - Ashish K Khanna
- Department of Anesthesiology, Section on Critical Care Medicine, School of Medicine, Wake Forest University, Atrium Health Wake Forest Baptist Medical Center, Winston-Salem, NC
| | - Jiapeng Huang
- Department of Anesthesiology and Perioperative Medicine, University of Louisville, Louisville, KY.
| |
Collapse
|
2
|
Masuda Y, Ishikawa R, Tanaka T, Aoyama G, Kawashima K, Chapman JV, Asami M, Pham MHC, Kofoed KF, Sakaguchi T, Satoh K. CNN-based fully automatic mitral valve extraction using CT images and existence probability maps. Phys Med Biol 2024; 69:035001. [PMID: 38100829 DOI: 10.1088/1361-6560/ad162b] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2023] [Accepted: 12/15/2023] [Indexed: 12/17/2023]
Abstract
Objective. Accurate extraction of mitral valve shape from clinical tomographic images acquired in patients has proven useful for planning surgical and interventional mitral valve treatments. However, manual extraction of the mitral valve shape is laborious, and the existing automatic extraction methods have not been sufficiently accurate. In this paper, we propose a fully automated method of extracting mitral valve shape from computed tomography (CT) images for the all phases of the cardiac cycle.Approach. This method extracts the mitral valve shape based on DenseNet using both the original CT image and the existence probability maps of the mitral valve area inferred by U-Net as input. A total of 1585 CT images from 204 patients with various cardiac diseases including mitral regurgitation were collected and manually annotated for mitral valve region. The proposed method was trained and evaluated by 10-fold cross validation using the collected data and was compared with the method without the existence probability maps.Main results. The mean error of shape extraction error in the proposed method is 0.88 mm, which is an improvement of 0.32 mm compared with the method without the existence probability maps.Significance. We present a novel fully automatic mitral valve extraction method from input to output for all phases of 4D CT images. We suggest that the accuracy of mitral valve shape extraction is improved by using existence probability maps.
Collapse
Affiliation(s)
- Yukiteru Masuda
- Canon Inc., 30-2, Shimomaruko 3-chome, Ohta-ku, Tokyo 146-8501, Japan
| | - Ryo Ishikawa
- Canon Inc., 30-2, Shimomaruko 3-chome, Ohta-ku, Tokyo 146-8501, Japan
| | - Toru Tanaka
- Canon Inc., 30-2, Shimomaruko 3-chome, Ohta-ku, Tokyo 146-8501, Japan
| | - Gakuto Aoyama
- Canon Medical Systems Corporation., 1385, Shimoishigami, Otawara-shi, Tochigi 324-8550, Japan
| | - Keitaro Kawashima
- Canon Medical Systems Corporation., 1385, Shimoishigami, Otawara-shi, Tochigi 324-8550, Japan
| | - James V Chapman
- Canon Medical Informatics, Inc. 5850 Opus Parkway, Suite 300 1 Minnetonka, MN 55343 1, United States of America
| | - Masahiko Asami
- Division of Cardiology, Mitsui Memorial Hospital, Kanda-Izumicho 1, Chiyoda-ku, Tokyo, 101-8643, Japan
| | - Michael Huy Cuong Pham
- Department of Cardiology and Radiology, Copenhagen University Hospital - Rigshospitalet & Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Blegdamsvej 9, DK-2100 Copenhagen, Denmark
| | - Klaus Fuglsang Kofoed
- Department of Cardiology and Radiology, Copenhagen University Hospital - Rigshospitalet & Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Blegdamsvej 9, DK-2100 Copenhagen, Denmark
| | - Takuya Sakaguchi
- Canon Medical Systems Corporation., 1385, Shimoishigami, Otawara-shi, Tochigi 324-8550, Japan
| | - Kiyohide Satoh
- Canon Inc., 30-2, Shimomaruko 3-chome, Ohta-ku, Tokyo 146-8501, Japan
| |
Collapse
|
3
|
Zha SZ, Rogstadkjernet M, Klæboe LG, Skulstad H, Singstad BJ, Gilbert A, Edvardsen T, Samset E, Brekke PH. Deep learning for automated left ventricular outflow tract diameter measurements in 2D echocardiography. Cardiovasc Ultrasound 2023; 21:19. [PMID: 37833731 PMCID: PMC10571406 DOI: 10.1186/s12947-023-00317-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/16/2023] [Accepted: 10/02/2023] [Indexed: 10/15/2023] Open
Abstract
BACKGROUND Measurement of the left ventricular outflow tract diameter (LVOTd) in echocardiography is a common source of error when used to calculate the stroke volume. The aim of this study is to assess whether a deep learning (DL) model, trained on a clinical echocardiographic dataset, can perform automatic LVOTd measurements on par with expert cardiologists. METHODS Data consisted of 649 consecutive transthoracic echocardiographic examinations of patients with coronary artery disease admitted to a university hospital. 1304 LVOTd measurements in the parasternal long axis (PLAX) and zoomed parasternal long axis views (ZPLAX) were collected, with each patient having 1-6 measurements per examination. Data quality control was performed by an expert cardiologist, and spatial geometry data was preserved for each LVOTd measurement to convert DL predictions into metric units. A convolutional neural network based on the U-Net was used as the DL model. RESULTS The mean absolute LVOTd error was 1.04 (95% confidence interval [CI] 0.90-1.19) mm for DL predictions on the test set. The mean relative LVOTd errors across all data subgroups ranged from 3.8 to 5.1% for the test set. Generally, the DL model had superior performance on the ZPLAX view compared to the PLAX view. DL model precision for patients with repeated LVOTd measurements had a mean coefficient of variation of 2.2 (95% CI 1.6-2.7) %, which was comparable to the clinicians for the test set. CONCLUSION DL for automatic LVOTd measurements in PLAX and ZPLAX views is feasible when trained on a limited clinical dataset. While the DL predicted LVOTd measurements were within the expected range of clinical inter-observer variability, the robustness of the DL model requires validation on independent datasets. Future experiments using temporal information and anatomical constraints could improve valvular identification and reduce outliers, which are challenges that must be addressed before clinical utilization.
Collapse
Affiliation(s)
| | | | | | - Helge Skulstad
- University of Oslo, Oslo, Norway
- Oslo University Hospital, Rikshospitalet, Oslo, Norway
| | | | | | - Thor Edvardsen
- University of Oslo, Oslo, Norway
- Oslo University Hospital, Rikshospitalet, Oslo, Norway
| | - Eigil Samset
- University of Oslo, Oslo, Norway
- GE HealthCare, Oslo, Norway
| | | |
Collapse
|
4
|
Taskén AA, Berg EAR, Grenne B, Holte E, Dalen H, Stølen S, Lindseth F, Aakhus S, Kiss G. Automated estimation of mitral annular plane systolic excursion by artificial intelligence from 3D ultrasound recordings. Artif Intell Med 2023; 144:102646. [PMID: 37783546 DOI: 10.1016/j.artmed.2023.102646] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2022] [Revised: 08/22/2023] [Accepted: 08/28/2023] [Indexed: 10/04/2023]
Abstract
Perioperative monitoring of cardiac function is beneficial for early detection of cardiovascular complications. The standard of care for cardiac monitoring performed by trained cardiologists and anesthesiologists involves a manual and qualitative evaluation of ultrasound imaging, which is a time-demanding and resource-intensive process with intraobserver- and interobserver variability. In practice, such measures can only be performed a limited number of times during the intervention. To overcome these difficulties, this study presents a robust method for automatic and quantitative monitoring of cardiac function based on 3D transesophageal echocardiography (TEE) B-mode ultrasound recordings of the left ventricle (LV). Such an assessment obtains consistent measurements and can produce a near real-time evaluation of ultrasound imagery. Hence, the presented method is time-saving and results in increased accessibility. The mitral annular plane systolic excursion (MAPSE), characterizing global LV function, is estimated by landmark detection and cardiac view classification of two-dimensional images extracted along the long-axis of the ultrasound volume. MAPSE estimation directly from 3D TEE recordings is beneficial since it removes the need for manual acquisition of cardiac views, hence decreasing the need for interference by physicians. Two convolutional neural networks (CNNs) were trained and tested on acquired ultrasound data of 107 patients, and MAPSE estimates were compared to clinically obtained references in a blinded study including 31 patients. The proposed method for automatic MAPSE estimation had low bias and low variability in comparison to clinical reference measures. The method accomplished a mean difference for MAPSE estimates of (-0.16±1.06) mm. Thus, the results did not show significant systematic errors. The obtained bias and variance of the method were comparable to inter-observer variability of clinically obtained MAPSE measures on 2D TTE echocardiography. The novel pipeline proposed in this study has the potential to enhance cardiac monitoring in perioperative- and intensive care settings.
Collapse
Affiliation(s)
- Anders Austlid Taskén
- Department of Computer Science, Faculty of Information Technology and Electrical Engineering, Norwegian University of Science and Technology, Høgskoleringen 1, 7491 Trondheim, Norway.
| | - Erik Andreas Rye Berg
- Department of Circulation and Medical Imaging, Faculty of Medicine and Health Sciences, Norwegian University of Science and Technology, Norway; Clinic of Cardiology, St. Olavs University Hospital, Trondheim, Norway.
| | - Bjørnar Grenne
- Department of Circulation and Medical Imaging, Faculty of Medicine and Health Sciences, Norwegian University of Science and Technology, Norway; Clinic of Cardiology, St. Olavs University Hospital, Trondheim, Norway.
| | - Espen Holte
- Department of Circulation and Medical Imaging, Faculty of Medicine and Health Sciences, Norwegian University of Science and Technology, Norway; Clinic of Cardiology, St. Olavs University Hospital, Trondheim, Norway.
| | - Håvard Dalen
- Department of Circulation and Medical Imaging, Faculty of Medicine and Health Sciences, Norwegian University of Science and Technology, Norway; Clinic of Cardiology, St. Olavs University Hospital, Trondheim, Norway; Levanger Hospital, Nord-Trøndelag Hospital Trust, Levanger, Norway.
| | - Stian Stølen
- Department of Circulation and Medical Imaging, Faculty of Medicine and Health Sciences, Norwegian University of Science and Technology, Norway; Clinic of Cardiology, St. Olavs University Hospital, Trondheim, Norway.
| | - Frank Lindseth
- Department of Computer Science, Faculty of Information Technology and Electrical Engineering, Norwegian University of Science and Technology, Høgskoleringen 1, 7491 Trondheim, Norway.
| | - Svend Aakhus
- Department of Circulation and Medical Imaging, Faculty of Medicine and Health Sciences, Norwegian University of Science and Technology, Norway; Clinic of Cardiology, St. Olavs University Hospital, Trondheim, Norway.
| | - Gabriel Kiss
- Department of Computer Science, Faculty of Information Technology and Electrical Engineering, Norwegian University of Science and Technology, Høgskoleringen 1, 7491 Trondheim, Norway.
| |
Collapse
|
5
|
Chen J, Li H, He G, Yao F, Lai L, Yao J, Xie L. Automatic 3D mitral valve leaflet segmentation and validation of quantitative measurement. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2022.104166] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/14/2022]
|
6
|
Gearhart A, Goto S, Deo RC, Powell AJ. An Automated View Classification Model for Pediatric Echocardiography Using Artificial Intelligence. J Am Soc Echocardiogr 2022; 35:1238-1246. [PMID: 36049595 PMCID: PMC9990955 DOI: 10.1016/j.echo.2022.08.009] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/13/2022] [Revised: 07/10/2022] [Accepted: 08/12/2022] [Indexed: 10/15/2022]
Abstract
BACKGROUND View classification is a key step toward building a fully automated system for interpretation of echocardiograms. However, compared with adult echocardiograms, creating a view classification model for pediatric echocardiograms poses additional challenges, such as greater variation in anatomy, structure size, and views. The aim of this study was to develop a computer vision model to autonomously perform view classification on pediatric echocardiographic images. METHODS Using a training set of 12,067 echocardiographic images from patients aged 0 to 19 years, a convolutional neural network model was trained to identify 27 preselected standard pediatric echocardiographic views which included anatomic sweeps, color Doppler, and Doppler tracings. A validation set of 6,197 images was used for parameter tuning and model selection. A test set of 9,684 images from 100 different patients was then used to evaluate model accuracy. The model was also evaluated on a per study basis using a second test set consisting of 524 echocardiograms from children with leukemia to identify six preselected views pertinent to cardiac dysfunction surveillance. RESULTS The model identified the 27 preselected views with 90.3% accuracy. Accuracy was similar across age groups (89.3% for 0-4 years, 90.8% for 4-9 years, 90.0% for 9-14 years, and 91.2% for 14-19 years; P = .12). Examining the view subtypes, accuracy was 78.3% for the cine one location, 90.5% for sweeps with color Doppler, 82.2% for sweeps without color Doppler, and 91.1% for Doppler tracings. Among the leukemia cohort, the model identified the six preselected views on a per study basis with a positive predictive value of 98.7% to 99.2% and sensitivity of 76.9% to 94.8%. CONCLUSIONS A convolutional neural network model was constructed for view classification of pediatric echocardiograms that was accurate across the spectrum of ages and view types. This work lays the foundation for automated quantitative analysis and diagnostic support to promote efficient, accurate, and scalable analysis of pediatric echocardiograms.
Collapse
Affiliation(s)
- Addison Gearhart
- Department of Cardiology, Boston Children's Hospital, and Department of Pediatrics, Harvard Medical School, Boston, Massachusetts.
| | - Shinichi Goto
- One Brave Idea, Division of Cardiovascular Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, Massachusetts
| | - Rahul C Deo
- One Brave Idea, Division of Cardiovascular Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, Massachusetts
| | - Andrew J Powell
- Department of Cardiology, Boston Children's Hospital, and Department of Pediatrics, Harvard Medical School, Boston, Massachusetts
| |
Collapse
|
7
|
Gautam N, Ghanta SN, Clausen A, Saluja P, Sivakumar K, Dhar G, Chang Q, DeMazumder D, Rabbat MG, Greene SJ, Fudim M, Al'Aref SJ. Contemporary Applications of Machine Learning for Device Therapy in Heart Failure. JACC. HEART FAILURE 2022; 10:603-622. [PMID: 36049812 DOI: 10.1016/j.jchf.2022.06.011] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/23/2022] [Revised: 05/31/2022] [Accepted: 06/16/2022] [Indexed: 06/15/2023]
Abstract
Despite a better understanding of the underlying pathogenesis of heart failure (HF), pharmacotherapy, surgical, and percutaneous interventions do not prevent disease progression in all patients, and a significant proportion of patients end up requiring advanced therapies. Machine learning (ML) is gaining wider acceptance in cardiovascular medicine because of its ability to incorporate large, complex, and multidimensional data and to potentially facilitate the creation of predictive models not constrained by many of the limitations of traditional statistical approaches. With the coexistence of "big data" and novel advanced analytic techniques using ML, there is ever-increasing research into applying ML in the context of HF with the goal of improving patient outcomes. Through this review, the authors describe the basics of ML and summarize the existing published reports regarding contemporary applications of ML in device therapy for HF while highlighting the limitations to widespread implementation and its future promises.
Collapse
Affiliation(s)
- Nitesh Gautam
- Department of Internal Medicine, University of Arkansas for Medical Sciences, Little Rock, Arkansas, USA
| | - Sai Nikhila Ghanta
- Department of Internal Medicine, University of Arkansas for Medical Sciences, Little Rock, Arkansas, USA
| | - Alex Clausen
- Department of Internal Medicine, University of Arkansas for Medical Sciences, Little Rock, Arkansas, USA
| | - Prachi Saluja
- Department of Internal Medicine, University of Arkansas for Medical Sciences, Little Rock, Arkansas, USA
| | - Kalai Sivakumar
- Division of Cardiology, Department of Medicine, University of Arkansas for Medical Sciences, Little Rock, Arkansas, USA
| | - Gaurav Dhar
- Division of Cardiology, Department of Medicine, University of Arkansas for Medical Sciences, Little Rock, Arkansas, USA
| | - Qi Chang
- Department of Computer Science, Rutgers University, The State University of New Jersey, Newark, New Jersey, USA
| | | | - Mark G Rabbat
- Department of Cardiology, Loyola University Medical Center, Maywood, Illinois, USA
| | - Stephen J Greene
- Department of Cardiology, Duke University Medical Center, Durham, North Carolina, USA; Duke Clinical Research Institute, Durham, North Carolina, USA
| | - Marat Fudim
- Department of Cardiology, Duke University Medical Center, Durham, North Carolina, USA; Duke Clinical Research Institute, Durham, North Carolina, USA
| | - Subhi J Al'Aref
- Division of Cardiology, Department of Medicine, University of Arkansas for Medical Sciences, Little Rock, Arkansas, USA.
| |
Collapse
|
8
|
Bølviken HS, Gerard O, Veronesi F, Samset E. Automatic alignment of standard views for transesophageal echocardiographic images. J Med Imaging (Bellingham) 2022; 9:057001. [PMID: 36330040 PMCID: PMC9618462 DOI: 10.1117/1.jmi.9.5.057001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/18/2022] [Accepted: 10/12/2022] [Indexed: 06/16/2023] Open
Abstract
Purpose 3D transesophageal echocardiography (TEE) has become an important modality for pre- and peri-operative imaging of valvular heart disease. TEE can give excellent visualization of valve morphology in 3D rendering. As a convention, 3D TEE images are reformatted in three standard views. We describe a method for automatic calculation of parameters needed to define the standard views from 3D TEE images using no manual input. Approach An algorithm was designed to find the center of the mitral valve and the left ventricular outflow tract (OT). These parameters defined the three-chamber view. The problem was modeled as a state estimation problem in which a 3D model was deformed based on shape priors and edge detection using a Kalman filter. This algorithm is capable of running in real time after initialization. Results The algorithm was validated by comparing the automatic alignments of 106 TEE images against manually placed landmarks. The median error for determining the mitral valve center was 7.1 mm, and the median error for determining the left ventricular OT orientation was 13.5 deg. Conclusion The algorithm is an accurate tool for automating the process of finding standard views for TEE images of the mitral valve.
Collapse
Affiliation(s)
| | | | | | - Eigil Samset
- University of Oslo, Oslo, Norway
- GE Vingmed Ultrasound, Oslo, Norway
| |
Collapse
|
9
|
Nedadur R, Wang B, Tsang W. Artificial intelligence for the echocardiographic assessment of valvular heart disease. Heart 2022; 108:1592-1599. [PMID: 35144983 PMCID: PMC9554049 DOI: 10.1136/heartjnl-2021-319725] [Citation(s) in RCA: 24] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/29/2021] [Accepted: 12/29/2021] [Indexed: 11/18/2022] Open
Abstract
Developments in artificial intelligence (AI) have led to an explosion of studies exploring its application to cardiovascular medicine. Due to the need for training and expertise, one area where AI could be impactful would be in the diagnosis and management of valvular heart disease. This is because AI can be applied to the multitude of data generated from clinical assessments, imaging and biochemical testing during the care of the patient. In the area of valvular heart disease, the focus of AI has been on the echocardiographic assessment and phenotyping of patient populations to identify high-risk groups. AI can assist image acquisition, view identification for review, and segmentation of valve and cardiac structures for automated analysis. Using image recognition algorithms, aortic and mitral valve disease states have been directly detected from the images themselves. Measurements obtained during echocardiographic valvular assessment have been integrated with other clinical data to identify novel aortic valve disease subgroups and describe new predictors of aortic valve disease progression. In the future, AI could integrate echocardiographic parameters with other clinical data for precision medical management of patients with valvular heart disease.
Collapse
Affiliation(s)
- Rashmi Nedadur
- Division of Cardiac Surgery, University of Toronto, Toronto, Ontario, Canada.,Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, Ontario, Canada
| | - Bo Wang
- Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, Ontario, Canada.,Department of Computer Science, University of Toronto, Toronto, Ontario, Canada.,Vector Institute of Artificial Intelligence, University of Toronto, Toronto, Ontario, Canada.,Peter Munk Cardiac Center, University Health Network, Toronto, Ontario, Canada
| | - Wendy Tsang
- Peter Munk Cardiac Center, University Health Network, Toronto, Ontario, Canada .,Division of Cardiology, University of Toronto, Toronto, Ontario, Canada
| |
Collapse
|
10
|
Automatic morphological classification of mitral valve diseases in echocardiographic images based on explainable deep learning methods. Int J Comput Assist Radiol Surg 2021; 17:413-425. [PMID: 34897594 DOI: 10.1007/s11548-021-02542-7] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2021] [Accepted: 11/30/2021] [Indexed: 10/19/2022]
Abstract
PURPOSE Carpentier's functional classification is a guide to explain the types of mitral valve regurgitation based on morphological features. There are four types of pathological morphologies, regardless of the presence or absence of mitral regurgitation: Type I, normal; Type II, mitral valve prolapse; Type IIIa, mitral valve stenosis; and Type IIIb, restricted mitral leaflet motion. The aim of this study was to automatically classify mitral valves using echocardiographic images. METHODS In our procedure, after the classification of apical 4-chamber (A4C) and parasternal long-axis (PLA) views, we extracted the systolic/diastolic phase of the cardiac cycle by calculating the left ventricular area. Six typical pre-trained models were fine-tuned with a 4-class model for the PLA and a 3-class model for the A4C views. As an additional contribution, to provide explainability, we applied the Gradient-weighted Class Activation Mapping (Grad-CAM) algorithm to visualize areas of echocardiographic images where the different models generated a prediction. RESULTS This approach conferred a proper understanding of where various networks "look" into echocardiographic images to predict the four types of pathological mitral valve morphologies. Considering the accuracy metric and Grad-CAM maps and by applying the Inception-ResNet-v2 architecture to classify Type II in the PLA view and ResNeXt50 architecture to classify the other three classes in the A4C view, we achieved an 80% rate of model accuracy in the test data set. CONCLUSIONS We suggest an explainable, fully automated, and rule-based procedure to classify the four types of mitral valve morphologies based on Carpentier's functional classification using deep learning on transthoracic echocardiographic images. Our study results infer the feasibility of the use of deep learning models to prepare quick and precise assessments of mitral valve morphologies in echocardiograms. According to our knowledge, our study is the first one that provides a public data set regarding the Carpentier classification of MV pathologies.
Collapse
|
11
|
Herz C, Pace DF, Nam HH, Lasso A, Dinh P, Flynn M, Cianciulli A, Golland P, Jolley MA. Segmentation of Tricuspid Valve Leaflets From Transthoracic 3D Echocardiograms of Children With Hypoplastic Left Heart Syndrome Using Deep Learning. Front Cardiovasc Med 2021; 8:735587. [PMID: 34957233 PMCID: PMC8696083 DOI: 10.3389/fcvm.2021.735587] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2021] [Accepted: 11/16/2021] [Indexed: 11/13/2022] Open
Abstract
Hypoplastic left heart syndrome (HLHS) is a severe congenital heart defect in which the right ventricle and associated tricuspid valve (TV) alone support the circulation. TV failure is thus associated with heart failure, and the outcome of TV valve repair are currently poor. 3D echocardiography (3DE) can generate high-quality images of the valve, but segmentation is necessary for precise modeling and quantification. There is currently no robust methodology for rapid TV segmentation, limiting the clinical application of these technologies to this challenging population. We utilized a Fully Convolutional Network (FCN) to segment tricuspid valves from transthoracic 3DE. We trained on 133 3DE image-segmentation pairs and validated on 28 images. We then assessed the effect of varying inputs to the FCN using Mean Boundary Distance (MBD) and Dice Similarity Coefficient (DSC). The FCN with the input of an annular curve achieved a median DSC of 0.86 [IQR: 0.81-0.88] and MBD of 0.35 [0.23-0.4] mm for the merged segmentation and an average DSC of 0.77 [0.73-0.81] and MBD of 0.6 [0.44-0.74] mm for individual TV leaflet segmentation. The addition of commissural landmarks improved individual leaflet segmentation accuracy to an MBD of 0.38 [0.3-0.46] mm. FCN-based segmentation of the tricuspid valve from transthoracic 3DE is feasible and accurate. The addition of an annular curve and commissural landmarks improved the quality of the segmentations with MBD and DSC within the range of human inter-user variability. Fast and accurate FCN-based segmentation of the tricuspid valve in HLHS may enable rapid modeling and quantification, which in the future may inform surgical planning. We are now working to deploy this network for public use.
Collapse
Affiliation(s)
- Christian Herz
- Children's Hospital of Philadelphia, Department of Anesthesia and Critical Care Medicine, Philadelphia, PA, United States
| | - Danielle F. Pace
- Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, United States
- Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School, Boston, MA, United States
| | - Hannah H. Nam
- Children's Hospital of Philadelphia, Department of Anesthesia and Critical Care Medicine, Philadelphia, PA, United States
| | - Andras Lasso
- Laboratory for Percutaneous Surgery, Queen's University, Kingston, ON, Canada
| | - Patrick Dinh
- Children's Hospital of Philadelphia, Department of Anesthesia and Critical Care Medicine, Philadelphia, PA, United States
| | - Maura Flynn
- Children's Hospital of Philadelphia, Department of Anesthesia and Critical Care Medicine, Philadelphia, PA, United States
| | - Alana Cianciulli
- Children's Hospital of Philadelphia, Department of Anesthesia and Critical Care Medicine, Philadelphia, PA, United States
| | - Polina Golland
- Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, United States
| | - Matthew A. Jolley
- Children's Hospital of Philadelphia, Department of Anesthesia and Critical Care Medicine, Philadelphia, PA, United States
- Division of Pediatric Cardiology, Children's Hospital of Philadelphia, Philadelphia, PA, United States
| |
Collapse
|
12
|
de Siqueira VS, Borges MM, Furtado RG, Dourado CN, da Costa RM. Artificial intelligence applied to support medical decisions for the automatic analysis of echocardiogram images: A systematic review. Artif Intell Med 2021; 120:102165. [PMID: 34629153 DOI: 10.1016/j.artmed.2021.102165] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2021] [Revised: 08/07/2021] [Accepted: 08/31/2021] [Indexed: 12/16/2022]
Abstract
The echocardiogram is a test that is widely used in Heart Disease Diagnoses. However, its analysis is largely dependent on the physician's experience. In this regard, artificial intelligence has become an essential technology to assist physicians. This study is a Systematic Literature Review (SLR) of primary state-of-the-art studies that used Artificial Intelligence (AI) techniques to automate echocardiogram analyses. Searches on the leading scientific article indexing platforms using a search string returned approximately 1400 articles. After applying the inclusion and exclusion criteria, 118 articles were selected to compose the detailed SLR. This SLR presents a thorough investigation of AI applied to support medical decisions for the main types of echocardiogram (Transthoracic, Transesophageal, Doppler, Stress, and Fetal). The article's data extraction indicated that the primary research interest of the studies comprised four groups: 1) Improvement of image quality; 2) identification of the cardiac window vision plane; 3) quantification and analysis of cardiac functions, and; 4) detection and classification of cardiac diseases. The articles were categorized and grouped to show the main contributions of the literature to each type of ECHO. The results indicate that the Deep Learning (DL) methods presented the best results for the detection and segmentation of the heart walls, right and left atrium and ventricles, and classification of heart diseases using images/videos obtained by echocardiography. The models that used Convolutional Neural Network (CNN) and its variations showed the best results for all groups. The evidence produced by the results presented in the tabulation of the studies indicates that the DL contributed significantly to advances in echocardiogram automated analysis processes. Although several solutions were presented regarding the automated analysis of ECHO, this area of research still has great potential for further studies to improve the accuracy of results already known in the literature.
Collapse
Affiliation(s)
- Vilson Soares de Siqueira
- Federal Institute of Tocantins, Av. Bernado Sayão, S/N, Santa Maria, Colinas do Tocantins, TO, Brazil; Federal University of Goias, Alameda Palmeiras, Quadra D, Câmpus Samambaia, Goiânia, GO, Brazil.
| | - Moisés Marcos Borges
- Diagnostic Imaging Center - CDI, Av. Portugal, 1155, St. Marista, Goiânia, GO, Brazil
| | - Rogério Gomes Furtado
- Diagnostic Imaging Center - CDI, Av. Portugal, 1155, St. Marista, Goiânia, GO, Brazil
| | - Colandy Nunes Dourado
- Diagnostic Imaging Center - CDI, Av. Portugal, 1155, St. Marista, Goiânia, GO, Brazil. http://www.cdigoias.com.br
| | - Ronaldo Martins da Costa
- Federal University of Goias, Alameda Palmeiras, Quadra D, Câmpus Samambaia, Goiânia, GO, Brazil.
| |
Collapse
|
13
|
Lauzier PT, Avram R, Dey D, Slomka P, Afilalo J, Chow BJ. The evolving role of artificial intelligence in cardiac image analysis. Can J Cardiol 2021; 38:214-224. [PMID: 34619340 DOI: 10.1016/j.cjca.2021.09.030] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/20/2021] [Revised: 09/28/2021] [Accepted: 09/28/2021] [Indexed: 12/13/2022] Open
Abstract
Research in artificial intelligence (AI) have progressed over the last decade. The field of cardiac imaging has seen significant developments using newly developed deep learning methods for automated image analysis and AI tools for disease detection and prognostication. This review article is aimed at those without special background in AI. We review AI concepts and we survey the growing contemporary applications of AI for image analysis in echocardiography, nuclear cardiology, cardiac computed tomography, cardiac magnetic resonance, and invasive angiography.
Collapse
Affiliation(s)
| | - Robert Avram
- University of Ottawa Heart Institute, Ottawa, ON, Canada; Montreal Heart Institute, Montreal, QC, Canada
| | - Damini Dey
- Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Piotr Slomka
- Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | | | | |
Collapse
|
14
|
Sermesant M, Delingette H, Cochet H, Jaïs P, Ayache N. Applications of artificial intelligence in cardiovascular imaging. Nat Rev Cardiol 2021; 18:600-609. [PMID: 33712806 DOI: 10.1038/s41569-021-00527-2] [Citation(s) in RCA: 55] [Impact Index Per Article: 18.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 02/08/2021] [Indexed: 01/31/2023]
Abstract
Research into artificial intelligence (AI) has made tremendous progress over the past decade. In particular, the AI-powered analysis of images and signals has reached human-level performance in many applications owing to the efficiency of modern machine learning methods, in particular deep learning using convolutional neural networks. Research into the application of AI to medical imaging is now very active, especially in the field of cardiovascular imaging because of the challenges associated with acquiring and analysing images of this dynamic organ. In this Review, we discuss the clinical questions in cardiovascular imaging that AI can be used to address and the principal methodological AI approaches that have been developed to solve the related image analysis problems. Some approaches are purely data-driven and rely mainly on statistical associations, whereas others integrate anatomical and physiological information through additional statistical, geometric and biophysical models of the human heart. In a structured manner, we provide representative examples of each of these approaches, with particular attention to the underlying computational imaging challenges. Finally, we discuss the remaining limitations of AI approaches in cardiovascular imaging (such as generalizability and explainability) and how they can be overcome.
Collapse
Affiliation(s)
| | | | - Hubert Cochet
- IHU Liryc, CHU Bordeaux, Université Bordeaux, Inserm 1045, Pessac, France
| | - Pierre Jaïs
- IHU Liryc, CHU Bordeaux, Université Bordeaux, Inserm 1045, Pessac, France
| | | |
Collapse
|
15
|
Silva B, Pessanha I, Correia-Pinto J, Fonseca JC, Queiros S. Automatic assessment of Pectus Excavatum severity from CT images using deep learning. IEEE J Biomed Health Inform 2021; 26:324-333. [PMID: 34152992 DOI: 10.1109/jbhi.2021.3090966] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
Pectus excavatum (PE) is the most common abnormality of the thoracic cage, whose severity is evaluated by extracting three indices (Haller, correction and asymmetry) from computed tomography (CT) images. To date, this analysis is performed manually, which is tedious and prone to variability. In this paper, a fully automatic framework for PE severity quantification from CT images is proposed, comprising three steps: (1) identification of the sternue's greatest depression point; (2) detection of 8 anatomical keypoints relevant for severity assessment; and (3) measurements' geometric regularization and extraction. The first two steps rely on heatmap regression networks based on the Unet++ architecture, including a novel variant adapted to predict 1D confidence maps. The framework was evaluated on a database with 269 CTs. For comparative purposes, intra-observer, inter-observer and intra-patient variability of the estimated indices were analyzed in a subset of patients. The developed system showed a good agreement with the manual approach (a mean relative absolute error of 4.41%, 5.22% and 1.86% for the Haller, correction, and asymmetry indices, respectively), with limits of agreement comparable to the inter-observer variability. In the intrapatient analysis, the proposed framework outperformed the expert, showing a higher reproducibility between indices extracted from distinct CTs of the same patient. Overall, these results support the feasibility of the developed framework for the automatic, accurate and reproducible quantification of PE severity in a clinical context.
Collapse
|
16
|
Dual attention enhancement feature fusion network for segmentation and quantitative analysis of paediatric echocardiography. Med Image Anal 2021; 71:102042. [PMID: 33784600 DOI: 10.1016/j.media.2021.102042] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2020] [Revised: 03/09/2021] [Accepted: 03/10/2021] [Indexed: 12/19/2022]
Abstract
Paediatric echocardiography is a standard method for screening congenital heart disease (CHD). The segmentation of paediatric echocardiography is essential for subsequent extraction of clinical parameters and interventional planning. However, it remains a challenging task due to (1) the considerable variation of key anatomic structures, (2) the poor lateral resolution affecting accurate boundary definition, (3) the existence of speckle noise and artefacts in echocardiographic images. In this paper, we propose a novel deep network to address these challenges comprehensively. We first present a dual-path feature extraction module (DP-FEM) to extract rich features via a channel attention mechanism. A high- and low-level feature fusion module (HL-FFM) is devised based on spatial attention, which selectively fuses rich semantic information from high-level features with spatial cues from low-level features. In addition, a hybrid loss is designed to deal with pixel-level misalignment and boundary ambiguities. Based on the segmentation results, we derive key clinical parameters for diagnosis and treatment planning. We extensively evaluate the proposed method on 4,485 two-dimensional (2D) paediatric echocardiograms from 127 echocardiographic videos. The proposed method consistently achieves better segmentation performance than other state-of-the-art methods, whichdemonstratesfeasibility for automatic segmentation and quantitative analysis of paediatric echocardiography. Our code is publicly available at https://github.com/end-of-the-century/Cardiac.
Collapse
|
17
|
|
18
|
Duran Karaduman B, Ayhan H, Keleş T, Bozkurt E. The playmaker of the mitral valve disease: Mitral annulus. Int J Cardiol 2020; 316:205-206. [PMID: 32387253 DOI: 10.1016/j.ijcard.2020.04.075] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/25/2020] [Revised: 04/22/2020] [Accepted: 04/24/2020] [Indexed: 11/19/2022]
Affiliation(s)
- Bilge Duran Karaduman
- Atılım University, Faculty of Medicine, Department of Cardiology, Medicana International Ankara Hospital, Turkey.
| | - Hüseyin Ayhan
- Atılım University, Faculty of Medicine, Department of Cardiology, Medicana International Ankara Hospital, Turkey.
| | - Telat Keleş
- Ankara Yıldırım Beyazıt University, Faculty of Medicine, Department of Cardiology, Ankara City Hospital, Turkey
| | - Engin Bozkurt
- Medicana International Ankara Hospital, Department of Cardiology, Turkey
| |
Collapse
|