1
|
van Smeden M, Heinze G, Van Calster B, Asselbergs FW, Vardas PE, Bruining N, de Jaegere P, Moore JH, Denaxas S, Boulesteix AL, Moons KGM. Critical appraisal of artificial intelligence-based prediction models for cardiovascular disease. Eur Heart J 2022; 43:2921-2930. [PMID: 35639667 PMCID: PMC9443991 DOI: 10.1093/eurheartj/ehac238] [Citation(s) in RCA: 48] [Impact Index Per Article: 24.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/09/2021] [Revised: 03/29/2022] [Accepted: 04/26/2022] [Indexed: 11/12/2022] Open
Abstract
The medical field has seen a rapid increase in the development of artificial intelligence (AI)-based prediction models. With the introduction of such AI-based prediction model tools and software in cardiovascular patient care, the cardiovascular researcher and healthcare professional are challenged to understand the opportunities as well as the limitations of the AI-based predictions. In this article, we present 12 critical questions for cardiovascular health professionals to ask when confronted with an AI-based prediction model. We aim to support medical professionals to distinguish the AI-based prediction models that can add value to patient care from the AI that does not.
Collapse
Affiliation(s)
- Maarten van Smeden
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Universiteitsweg 100, 3584 CG Utrecht, The Netherlands
| | - Georg Heinze
- Section for Clinical Biometrics, Center for Medical Statistics, Informatics and Intelligent Systems, Medical University of Vienna, Vienna, Austria
| | - Ben Van Calster
- Department of Development and Regeneration, KU Leuven, Leuven, Belgium
- EPI Centre, KU Leuven, Leuven, Belgium
- Department of Biomedical Data Sciences, Leiden University Medical Centre, Leiden, The Netherlands
| | - Folkert W Asselbergs
- Department of Cardiology, Division Heart and Lungs, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
- Institute of Cardiovascular Science, Faculty of Population Health Sciences, University College London, London, UK
- Health Data Research UK and Institute of Health Informatics, University College London, London, UK
| | - Panos E Vardas
- Department of Cardiology, Heraklion University Hospital, Heraklion, Greece
- Heart Sector, Hygeia Hospitals Group, Athens, Greece
| | - Nico Bruining
- Department of Cardiology, Erasmus MC , Thorax Center, Rotterdam, The Netherlands
| | - Peter de Jaegere
- Department of Cardiology, Erasmus MC, Thorax Center, Rotterdam, The Netherlands
| | - Jason H Moore
- Department of Computational Biomedicine, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Spiros Denaxas
- Health Data Research UK and Institute of Health Informatics, University College London, London, UK
- The Alan Turing Institute, London, UK
| | - Anne Laure Boulesteix
- Institute for Medical Information Processing, Biometry and Epidemiology, LMU Munich, Germany
| | - Karel G M Moons
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Universiteitsweg 100, 3584 CG Utrecht, The Netherlands
| |
Collapse
|
2
|
Nijpels G, van der Heijden AAWA, Elders P, Beulens JWJ, de Vet HCW. The interobserver agreement of ECG abnormalities using Minnesota codes in people with type 2 diabetes. PLoS One 2021; 16:e0255466. [PMID: 34383817 PMCID: PMC8360582 DOI: 10.1371/journal.pone.0255466] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2020] [Accepted: 07/09/2021] [Indexed: 12/02/2022] Open
Abstract
OBJECTIVES To assess the interobserver agreement in categories of electrocardiogram (ECG) abnormalities using the Minnesota Code criteria. METHODS We used a random sample of 180 ECGs from people with type 2 diabetes. ECG abnormalities were classified and coded using the Minnesota ECG Classification. Each ECG was independently rated on several abnormalities by an experienced rater (rater 1) and by two cardiologists (raters 2 and 3) trained to apply the Minnesota codes on four Minnesota codes; 1-codes as an indication for myocardial infarction, 4 en 5-codes as an indication for ischemic abnormalities, 3-codes as an indication for left ventricle hypertrophy, 7-1-codes as an indication for ventricular conduction abnormalities, and 8-3-codes as an indication for atrial fibrillation / atrial flutter. After all pairwise tables were summed, the overall agreement, the specific positive and negative agreement were calculated with a 95% confidence interval (CI) for each abnormality. Also, Kappa's with a 95% CI were calculated. RESULTS The overall agreement (with 95% CI) were for myocardial infarction, ischemic abnormalities, left ventricle hypertrophy, conduction abnormalities and atrial fibrillation/atrial flutter respectively: 0.87 (0.84-0.91), 0.79 (0.74-0.84), 0.81 (0.76-0.85), 0.93 (0.90-0.95), 0.96 (0.93-0.97). CONCLUSION This study shows that the overall agreement of the Minnesota code is good to excellent.
Collapse
Affiliation(s)
- Giel Nijpels
- Department of General Practice and Elderly Care Medicine, Amsterdam University Medical Center, Location VU, Amsterdam, The Netherlands
| | - Amber A. W. A. van der Heijden
- Department of General Practice and Elderly Care Medicine, Amsterdam University Medical Center, Location VU, Amsterdam, The Netherlands
| | - Petra Elders
- Department of General Practice and Elderly Care Medicine, Amsterdam University Medical Center, Location VU, Amsterdam, The Netherlands
| | - Joline W. J. Beulens
- Department of Epidemiology and Data Science, Amsterdam University Medical Center, Location VU, Amsterdam, The Netherlands
| | - Henrica C. W. de Vet
- Department of Epidemiology and Data Science, Amsterdam University Medical Center, Location VU, Amsterdam, The Netherlands
| |
Collapse
|
3
|
Wittwer MR, Zeitz C, Wu S, Mishra K, Rajendran S, Beltrame JF, Arstall MA. Cardiologists appropriately exclude resuscitated out-of-hospital cardiac arrests from emergency coronary angiography. J Am Coll Emerg Physicians Open 2020; 1:1177-1184. [PMID: 33392520 PMCID: PMC7771780 DOI: 10.1002/emp2.12276] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2020] [Revised: 09/20/2020] [Accepted: 09/22/2020] [Indexed: 11/20/2022] Open
Abstract
OBJECTIVE Emergency coronary angiography after resuscitated out-of-hospital cardiac arrest as a selective or non-selective diagnostic procedure with or without intervention continues to be the subject of debate. This study sought to determine if cardiologists reliably select patients using clinical judgement for emergency coronary angiography without missing acutely ischemic cases requiring revascularization. METHODS Presenting clinical details and ECGs (within 2 hours) from 52 consecutive out-of-hospital cardiac arrest patients who underwent non-selective coronary angiography were compiled retrospectively. Three out-of-hospital cardiac arrest-experienced interventional cardiologists, blinded to patient outcome, independently determined working diagnosis, and decision for emergency coronary angiography using clinical judgement. Sensitivity of the cardiologists' decision was assessed with respect to the outcome of acute revascularization. Inter-rater differences, consensus in clinical assessment, and influence of working diagnosis were also investigated. RESULTS Sensitivity of individual cardiologist's decision for emergency coronary angiography with respect to acute revascularization was very high (adjusted overall sensitivity = 95.8%, 95% CI = 89-100, cardiologist range = 93%-100%), and perfect for the consensus of 2 or more cardiologists (100%, 95% CI = 79.4-100). There was no statistical difference in the sensitivity of this decision between cardiologists (P < 0.05), and inter-rater agreement was moderate (78% overall agreement, Κ = 0.56). CONCLUSIONS Experienced cardiologists recommend emergency coronary angiography in all resuscitated out-of-hospital cardiac arrest requiring acute revascularization and appropriately excluded one-third of patients. Rather than advocating a non-selective, or conversely, a restrictive strategy with respect to coronary angiography after out-of-hospital cardiac arrest, the findings support an individualized approach by a multidisciplinary emergency team that includes experienced cardiologists. The results should be confirmed in a larger prospective study.
Collapse
Affiliation(s)
- Melanie R. Wittwer
- School of MedicineUniversity of AdelaideAdelaideSouth AustraliaAustralia
- Cardiology UnitLyell McEwin and Modbury Hospitals, Northern Adelaide Local Health NetworkElizabeth ValeSouth AustraliaAustralia
| | - Chris Zeitz
- School of MedicineUniversity of AdelaideAdelaideSouth AustraliaAustralia
- Cardiology UnitThe Queen Elizabeth and Royal Adelaide Hospitals, Central Adelaide Local Health NetworkWoodville SouthSouth AustraliaAustralia
| | - Sunny Wu
- Princess Alexandra HospitalWoolloongabbaQueenslandAustralia
| | - Kumaril Mishra
- Cardiology UnitLyell McEwin and Modbury Hospitals, Northern Adelaide Local Health NetworkElizabeth ValeSouth AustraliaAustralia
| | - Sharmalar Rajendran
- School of MedicineUniversity of AdelaideAdelaideSouth AustraliaAustralia
- Cardiology UnitLyell McEwin and Modbury Hospitals, Northern Adelaide Local Health NetworkElizabeth ValeSouth AustraliaAustralia
| | - John F. Beltrame
- School of MedicineUniversity of AdelaideAdelaideSouth AustraliaAustralia
- Cardiology UnitThe Queen Elizabeth and Royal Adelaide Hospitals, Central Adelaide Local Health NetworkWoodville SouthSouth AustraliaAustralia
| | - Margaret A. Arstall
- School of MedicineUniversity of AdelaideAdelaideSouth AustraliaAustralia
- Cardiology UnitLyell McEwin and Modbury Hospitals, Northern Adelaide Local Health NetworkElizabeth ValeSouth AustraliaAustralia
| |
Collapse
|
4
|
Leivo J, Anttonen E, Jolly SS, Dzavik V, Koivumäki J, Tahvanainen M, Koivula K, Nikus K, Wang J, Cairns JA, Niemelä K, Eskola MJ. The high-risk ECG pattern of ST-elevation myocardial infarction: A substudy of the randomized trial of primary PCI with or without routine manual thrombectomy (TOTAL trial). Int J Cardiol 2020; 319:40-45. [PMID: 32470531 DOI: 10.1016/j.ijcard.2020.05.053] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/03/2020] [Revised: 05/14/2020] [Accepted: 05/18/2020] [Indexed: 10/24/2022]
Abstract
BACKGROUND Useful tools for risk assessment in patients with STEMI are needed. We evaluated the prognostic impact of the evolving myocardial infarction (EMI) and the preinfarction syndrome (PIS) ECG patterns and determined their correlation with angiographic findings and treatment strategy. METHODS This substudy of the randomized Trial of Routine Aspiration Thrombectomy with PCI versus PCI Alone in Patients with STEMI (TOTAL) included 7860 patients with STEMI and either the EMI or the PIS ECG pattern. The primary outcome was a composite of death from cardiovascular causes, recurrent MI, cardiogenic shock, or New York Heart Association (NYHA) class IV heart failure within one year. RESULTS The primary outcome occurred in 271 of 2618 patients (10.4%) in the EMI group vs. 322 of 5242 patients (6.1%) in the PIS group [AdjustedHR, 1.54; 95% CI, 1.30 to 1.82; p < .001]. The primary outcome occurred in the thrombectomy and PCI alone groups in 131 of 1306 (10.0%) and 140 of 1312 (10.7%) patients with EMI [HR 0.94; 95% CI, 0.74-1.19] and 162 of 2633 (6.2%) and 160 of 2609 (6.1%) patients with PIS [HR 1.00; 95% CI, 0.81-1.25], respectively (pinteraction = 0.679). CONCLUSIONS Patients with the EMI ECG pattern proved to have an increased rate of the primary outcome within one year compared to the PIS pattern. Routine manual thrombectomy did not reduce the risk of primary outcome within the different dynamic ECG patterns. The PIS/EMI dynamic ECG classification could help to triage patients in case of simultaneous STEMI patients with immediate need for pPCI.
Collapse
Affiliation(s)
- Joonas Leivo
- Heart Center, Department of Cardiology, Tampere University Hospital, Finland; Faculty of Medicine and Health Technology, Tampere University, Finnish Cardiovascular Research Center, Tampere, Finland.
| | - Eero Anttonen
- Heart Center, Department of Cardiology, Tampere University Hospital, Finland; Faculty of Medicine and Health Technology, Tampere University, Finnish Cardiovascular Research Center, Tampere, Finland
| | - Sanjit S Jolly
- Population Health Research Institute, Hamilton, Canada; Department of Medicine, McMaster University, Hamilton, Canada; Hamilton Health Sciences, Hamilton, Canada
| | - Vladimir Dzavik
- Peter Munk Cardiac Centre, University Health Network, Toronto, Canada
| | - Jyri Koivumäki
- Heart Center, Department of Cardiology, Tampere University Hospital, Finland
| | - Minna Tahvanainen
- Heart Center, Department of Cardiology, Tampere University Hospital, Finland
| | - Kimmo Koivula
- Faculty of Medicine and Health Technology, Tampere University, Finnish Cardiovascular Research Center, Tampere, Finland; Internal medicine, Helsinki University Hospital, Finland
| | - Kjell Nikus
- Heart Center, Department of Cardiology, Tampere University Hospital, Finland; Faculty of Medicine and Health Technology, Tampere University, Finnish Cardiovascular Research Center, Tampere, Finland
| | - Jia Wang
- Population Health Research Institute, Hamilton, Canada; Department of Medicine, McMaster University, Hamilton, Canada; David Braley Cardiac, Vascular and Stroke Research Institute, Hamilton, Canada
| | | | - Kari Niemelä
- Heart Center, Department of Cardiology, Tampere University Hospital, Finland
| | - Markku J Eskola
- Heart Center, Department of Cardiology, Tampere University Hospital, Finland; Faculty of Medicine and Health Technology, Tampere University, Finnish Cardiovascular Research Center, Tampere, Finland
| |
Collapse
|
5
|
Novotny T, Bond RR, Andrsova I, Koc L, Sisakova M, Finlay DD, Guldenring D, Spinar J, Malik M. Data analysis of diagnostic accuracies in 12-lead electrocardiogram interpretation by junior medical fellows. J Electrocardiol 2015; 48:988-94. [DOI: 10.1016/j.jelectrocard.2015.08.023] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/21/2015] [Indexed: 10/23/2022]
|