1
|
Kagiyama N, Abe Y, Kusunose K, Kato N, Kaneko T, Murata A, Ota M, Shibayama K, Izumo M, Watanabe H. Multicenter validation study for automated left ventricular ejection fraction assessment using a handheld ultrasound with artificial intelligence. Sci Rep 2024; 14:15359. [PMID: 38965290 PMCID: PMC11224326 DOI: 10.1038/s41598-024-65557-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2023] [Accepted: 06/20/2024] [Indexed: 07/06/2024] Open
Abstract
We sought to validate the ability of a novel handheld ultrasound device with an artificial intelligence program (AI-POCUS) that automatically assesses left ventricular ejection fraction (LVEF). AI-POCUS was used to prospectively scan 200 patients in two Japanese hospitals. Automatic LVEF by AI-POCUS was compared to the standard biplane disk method using high-end ultrasound machines. After excluding 18 patients due to infeasible images for AI-POCUS, 182 patients (63 ± 15 years old, 21% female) were analyzed. The intraclass correlation coefficient (ICC) between the LVEF by AI-POCUS and the standard methods was good (0.81, p < 0.001) without clinically meaningful systematic bias (mean bias -1.5%, p = 0.008, limits of agreement ± 15.0%). Reduced LVEF < 50% was detected with a sensitivity of 85% (95% confidence interval 76%-91%) and specificity of 81% (71%-89%). Although the correlations between LV volumes by standard-echo and those by AI-POCUS were good (ICC > 0.80), AI-POCUS tended to underestimate LV volumes for larger LV (overall bias 42.1 mL for end-diastolic volume). These trends were mitigated with a newer version of the software tuned using increased data involving larger LVs, showing similar correlations (ICC > 0.85). In this real-world multicenter study, AI-POCUS showed accurate LVEF assessment, but careful attention might be necessary for volume assessment. The newer version, trained with larger and more heterogeneous data, demonstrated improved performance, underscoring the importance of big data accumulation in the field.
Collapse
Affiliation(s)
- Nobuyuki Kagiyama
- Department of Cardiovascular Biology and Medicine, Juntendo University Graduate School of Medicine, 2-1-1 Hongo, Tokyo, 113-0021, Japan.
- Department of Digital Health and Telemedicine R&D, Juntendo University, Tokyo, Japan.
| | - Yukio Abe
- Department of Cardiology, Osaka City General Hospital, Osaka, Japan
| | - Kenya Kusunose
- Department of Cardiovascular Medicine, Nephrology, and Neurology, University of the Ryukyus, Okinawa, Japan
| | - Nahoko Kato
- Department of Cardiology, Tokyo Bay Urayasu Ichikawa Medical Center, Urayasu, Japan
| | - Tomohiro Kaneko
- Department of Cardiovascular Biology and Medicine, Juntendo University Graduate School of Medicine, 2-1-1 Hongo, Tokyo, 113-0021, Japan
| | - Azusa Murata
- Department of Cardiovascular Biology and Medicine, Juntendo University Graduate School of Medicine, 2-1-1 Hongo, Tokyo, 113-0021, Japan
| | - Mitsuhiko Ota
- Department of Cardiovascular Center, Toranomon Hospital, Tokyo, Japan
| | - Kentaro Shibayama
- Department of Cardiovascular Medicine, Tokyo Cardiovascular and Internal Medicine Clinic, Tokyo, Japan
| | - Masaki Izumo
- Division of Cardiology, Department of Internal Medicine, St. Marianna University School of Medicine, Kawasaki, Japan
| | - Hiroyuki Watanabe
- Department of Cardiology, Tokyo Bay Urayasu Ichikawa Medical Center, Urayasu, Japan
| |
Collapse
|
2
|
Rindal OMH, Bjastad TG, Espeland T, Berg EAR, Masoy SE. A Very Large Cardiac Channel Data Database (VLCD) Used to Evaluate Global Image Coherence (GIC) as an In Vivo Image Quality Metric. IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2023; 70:1295-1307. [PMID: 37610900 DOI: 10.1109/tuffc.2023.3308034] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/25/2023]
Abstract
Ultrasound image quality is of utmost importance for a clinician to reach a correct diagnosis. Conventionally, image quality is evaluated using metrics to determine the contrast and resolution. These metrics require localization of specific regions and targets in the image such as a region of interest (ROI), a background region, and/or a point scatterer. Such objects can all be difficult to identify in in-vivo images, especially for automatic evaluation of image quality in large amounts of data. Using a matrix array probe, we have recorded a Very Large cardiac Channel data Database (VLCD) to evaluate coherence as an in vivo image quality metric. The VLCD consists of 33280 individual image frames from 538 recordings of 106 patients. We also introduce a global image coherence (GIC), an in vivo image quality metric that does not require any identified ROI since it is defined as an average coherence value calculated from all the data pixels used to form the image, below a preselected range. The GIC is shown to be a quantitative metric for in vivo image quality when applied to the VLCD. We demonstrate, on a subset of the dataset, that the GIC correlates well with the conventional metrics contrast ratio (CR) and the generalized contrast-to-noise ratio (gCNR) with R = 0.74 ( ) and R = 0.62 ( ), respectively. There exist multiple methods to estimate the coherence of the received signal across the ultrasound array. We further show that all coherence measures investigated in this study are highly correlated ( 0.9 and ) when applied to the VLCD. Thus, even though there are differences in the implementation of coherence measures, all quantify the similarity of the signal across the array and can be averaged into a GIC to evaluate image quality automatically and quantitatively.
Collapse
|
3
|
Sabo S, Pasdeloup D, Pettersen HN, Smistad E, Østvik A, Olaisen SH, Stølen SB, Grenne BL, Holte E, Lovstakken L, Dalen H. Real-time guidance by deep learning of experienced operators to improve the standardization of echocardiographic acquisitions. EUROPEAN HEART JOURNAL. IMAGING METHODS AND PRACTICE 2023; 1:qyad040. [PMID: 39045079 PMCID: PMC11195719 DOI: 10.1093/ehjimp/qyad040] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/11/2023] [Accepted: 11/22/2023] [Indexed: 07/25/2024]
Abstract
Aims Impaired standardization of echocardiograms may increase inter-operator variability. This study aimed to determine whether the real-time guidance of experienced sonographers by deep learning (DL) could improve the standardization of apical recordings. Methods and results Patients (n = 88) in sinus rhythm referred for echocardiography were included. All participants underwent three examinations, whereof two were performed by sonographers and the third by cardiologists. In the first study period (Period 1), the sonographers were instructed to provide echocardiograms for the analyses of the left ventricular function. Subsequently, after brief training, the DL guidance was used in Period 2 by the sonographer performing the second examination. View standardization was quantified retrospectively by a human expert as the primary endpoint and the DL algorithm as the secondary endpoint. All recordings were scored in rotation and tilt both separately and combined and were categorized as standardized or non-standardized. Sonographers using DL guidance had more standardized acquisitions for the combination of rotation and tilt than sonographers without guidance in both periods (all P ≤ 0.05) when evaluated by the human expert and DL [except for the apical two-chamber (A2C) view by DL evaluation]. When rotation and tilt were analysed individually, A2C and apical long-axis rotation and A2C tilt were significantly improved, and the others were numerically improved when evaluated by the echocardiography expert. Furthermore, all, except for A2C rotation, were significantly improved when evaluated by DL (P < 0.01). Conclusion Real-time guidance by DL improved the standardization of echocardiographic acquisitions by experienced sonographers. Future studies should evaluate the impact with respect to variability of measurements and when used by less-experienced operators. ClinicalTrialsgov Identifier NCT04580095.
Collapse
Affiliation(s)
- Sigbjorn Sabo
- Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, PO Box 8905, 7491 Trondheim, Norway
- Clinic of Cardiology, St.Olavs University Hospital, Prinsesse Kristinas gate 3, 7030 Trondheim, Norway
| | - David Pasdeloup
- Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, PO Box 8905, 7491 Trondheim, Norway
| | - Hakon Neergaard Pettersen
- Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, PO Box 8905, 7491 Trondheim, Norway
- Kristiansund Hospital, More and Romsdal Hospital Trust, Herman Døhlens veg 1, 6508 Kristiansund, Norway
| | - Erik Smistad
- Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, PO Box 8905, 7491 Trondheim, Norway
- Sintef Digital, Strindvegen 4, 7034 Trondheim, Norway
| | - Andreas Østvik
- Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, PO Box 8905, 7491 Trondheim, Norway
- Sintef Digital, Strindvegen 4, 7034 Trondheim, Norway
| | - Sindre Hellum Olaisen
- Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, PO Box 8905, 7491 Trondheim, Norway
| | - Stian Bergseng Stølen
- Clinic of Cardiology, St.Olavs University Hospital, Prinsesse Kristinas gate 3, 7030 Trondheim, Norway
| | - Bjørnar Leangen Grenne
- Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, PO Box 8905, 7491 Trondheim, Norway
- Clinic of Cardiology, St.Olavs University Hospital, Prinsesse Kristinas gate 3, 7030 Trondheim, Norway
| | - Espen Holte
- Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, PO Box 8905, 7491 Trondheim, Norway
- Clinic of Cardiology, St.Olavs University Hospital, Prinsesse Kristinas gate 3, 7030 Trondheim, Norway
| | - Lasse Lovstakken
- Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, PO Box 8905, 7491 Trondheim, Norway
| | - Havard Dalen
- Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, PO Box 8905, 7491 Trondheim, Norway
- Clinic of Cardiology, St.Olavs University Hospital, Prinsesse Kristinas gate 3, 7030 Trondheim, Norway
- Department of Internal Medicine, Levanger Hospital, Nord-Trøndelag Hospital Trust, Kirkegata 2, 7601 Levanger, Norway
| |
Collapse
|
4
|
Magelssen MI, Hjorth-Hansen AK, Andersen GN, Graven T, Kleinau JO, Skjetne K, Løvstakken L, Dalen H, Mjølstad OC. Clinical Influence of Handheld Ultrasound, Supported by Automatic Quantification and Telemedicine, in Suspected Heart Failure. ULTRASOUND IN MEDICINE & BIOLOGY 2023; 49:1137-1144. [PMID: 36804210 DOI: 10.1016/j.ultrasmedbio.2022.12.015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/19/2022] [Revised: 11/18/2022] [Accepted: 12/22/2022] [Indexed: 05/11/2023]
Abstract
Early and correct heart failure (HF) diagnosis is essential to improvement of patient care. We aimed to evaluate the clinical influence of handheld ultrasound device (HUD) examinations by general practitioners (GPs) in patients with suspected HF with or without the use of automatic measurement of left ventricular (LV) ejection fraction (autoEF), mitral annular plane systolic excursion (autoMAPSE) and telemedical support. Five GPs with limited ultrasound experience examined 166 patients with suspected HF (median interquartile range = 70 (63-78) y; mean ± SD EF = 53 ± 10%). They first performed a clinical examination. Second, they added an examination with HUD, automatic quantification tools and, finally, telemedical support by an external cardiologist. At all stages, the GPs considered whether the patients had HF. The final diagnosis was made by one of five cardiologists using medical history and clinical evaluation including a standard echocardiography. Compared with the cardiologists' decision, the GPs correctly classified 54% by clinical evaluation. The proportion increased to 71% after adding HUDs, and to 74 % after telemedical evaluation. Net reclassification improvement was highest for HUD with telemedicine. There was no significant benefit of the automatic tools (p ≥ 0.58). Addition of HUD and telemedicine improved the GPs' diagnostic precision in suspected HF. Automatic LV quantification added no benefit. Refined algorithms and more training may be needed before inexperienced users benefit from automatic quantification of cardiac function by HUDs.
Collapse
Affiliation(s)
- Malgorzata Izabela Magelssen
- Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, Trondheim, Norway; Clinic of Cardiology, St. Olavs University Hospital, Trondheim, Norway.
| | - Anna Katarina Hjorth-Hansen
- Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, Trondheim, Norway; Department of Internal Medicine, Levanger Hospital, Nord-Trøndelag Hospital Trust, Levanger, Norway
| | - Garrett Newton Andersen
- Department of Internal Medicine, Levanger Hospital, Nord-Trøndelag Hospital Trust, Levanger, Norway
| | - Torbjørn Graven
- Department of Internal Medicine, Levanger Hospital, Nord-Trøndelag Hospital Trust, Levanger, Norway
| | - Jens Olaf Kleinau
- Department of Internal Medicine, Levanger Hospital, Nord-Trøndelag Hospital Trust, Levanger, Norway
| | - Kyrre Skjetne
- Department of Internal Medicine, Levanger Hospital, Nord-Trøndelag Hospital Trust, Levanger, Norway
| | - Lasse Løvstakken
- Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, Trondheim, Norway
| | - Håvard Dalen
- Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, Trondheim, Norway; Clinic of Cardiology, St. Olavs University Hospital, Trondheim, Norway; Department of Internal Medicine, Levanger Hospital, Nord-Trøndelag Hospital Trust, Levanger, Norway
| | - Ole Christian Mjølstad
- Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, Trondheim, Norway; Clinic of Cardiology, St. Olavs University Hospital, Trondheim, Norway
| |
Collapse
|
5
|
Sabo S, Pettersen HN, Smistad E, Pasdeloup D, Stølen SB, Grenne BL, Lovstakken L, Holte E, Dalen H. Real-time guiding by deep learning during echocardiography to reduce left ventricular foreshortening and measurement variability. EUROPEAN HEART JOURNAL. IMAGING METHODS AND PRACTICE 2023; 1:qyad012. [PMID: 39044792 PMCID: PMC11195768 DOI: 10.1093/ehjimp/qyad012] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 05/12/2023] [Accepted: 07/20/2023] [Indexed: 07/25/2024]
Abstract
Aims Apical foreshortening leads to an underestimation of left ventricular (LV) volumes and an overestimation of LV ejection fraction and global longitudinal strain. Real-time guiding using deep learning (DL) during echocardiography to reduce foreshortening could improve standardization and reduce variability. We aimed to study the effect of real-time DL guiding during echocardiography on measures of LV foreshortening and inter-observer variability. Methods and results Patients (n = 88) in sinus rhythm referred for echocardiography without indication for contrast were included. All participants underwent three echocardiograms. The first two examinations were performed by sonographers, and the third by cardiologists. In Period 1, the sonographers were instructed to provide high-quality echocardiograms. In Period 2, the DL guiding was used by the second sonographer. One blinded expert measured LV length in all recordings. Tri-plane recordings by cardiologists were used as reference. Apical foreshortening was calculated at the end-diastole. Both sonographer groups significantly foreshortened the LV in Period 1 (mean foreshortening: Sonographer 1: 4 mm; Sonographer 2: 3 mm, both P < 0.001 vs. reference) and reduced foreshortening in Period 2 (2 and 0 mm, respectively. Period 1 vs. Period 2, P < 0.05). Sonographers using DL guiding did not foreshorten more than cardiologists (P ≥ 0.409). Real-time guiding did not improve intra-class correlation (ICC) [LV end-diastolic volume ICC, (95% confidence interval): DL guiding 0.87 (0.77-0.93) vs. no guiding 0.92 (0.88-0.95)]. Conclusion Real-time guiding reduced foreshortening among experienced operators and has the potential to improve image standardization. Even though the effect on inter-operator variability was minimal among experienced users, real-time guiding may improve test-retest variability among less experienced users. Clinical trial registration ClinicalTrials.gov, Identifier: NCT04580095.
Collapse
Affiliation(s)
- Sigbjorn Sabo
- Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, NTNU, Box 8905, 7491 Trondheim, Norway
| | - Hakon Neergaard Pettersen
- Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, NTNU, Box 8905, 7491 Trondheim, Norway
- Department of Internal Medicine, Kristiansund Hospital, More and Romsdal Hospital Trust, Herman Døhlens vei 1, 6508 Kristiansund, Norway
| | - Erik Smistad
- Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, NTNU, Box 8905, 7491 Trondheim, Norway
- Sintef Digital, Box 4760 Torgarden, 7465 Trondheim, Norway
| | - David Pasdeloup
- Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, NTNU, Box 8905, 7491 Trondheim, Norway
| | - Stian Bergseng Stølen
- Clinic of Cardiology, St Olavs University Hospital, Box 3250 Torgarden, 7006 Trondheim, Norway
| | - Bjørnar Leangen Grenne
- Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, NTNU, Box 8905, 7491 Trondheim, Norway
- Clinic of Cardiology, St Olavs University Hospital, Box 3250 Torgarden, 7006 Trondheim, Norway
| | - Lasse Lovstakken
- Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, NTNU, Box 8905, 7491 Trondheim, Norway
| | - Espen Holte
- Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, NTNU, Box 8905, 7491 Trondheim, Norway
- Clinic of Cardiology, St Olavs University Hospital, Box 3250 Torgarden, 7006 Trondheim, Norway
| | - Havard Dalen
- Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, NTNU, Box 8905, 7491 Trondheim, Norway
- Clinic of Cardiology, St Olavs University Hospital, Box 3250 Torgarden, 7006 Trondheim, Norway
- Department of Internal Medicine, Levanger Hospital, Nord-Trøndelag Hospital Trust, Kirkegata 2, 7601 Levanger, Norway
| |
Collapse
|