1
|
Raghunath S, Pfeifer JM, Kelsey CR, Nemani A, Ruhl JA, Hartzel DN, Ulloa Cerna AE, Jing L, vanMaanen DP, Leader JB, Schneider G, Morland TB, Chen R, Zimmerman N, Fornwalt BK, Haggerty CM. An ECG-based machine learning model for predicting new-onset atrial fibrillation is superior to age and clinical features in identifying patients at high stroke risk. J Electrocardiol 2023; 76:61-65. [PMID: 36436476 DOI: 10.1016/j.jelectrocard.2022.11.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/14/2022] [Revised: 10/11/2022] [Accepted: 11/02/2022] [Indexed: 11/09/2022]
Abstract
BACKGROUND Several large trials have employed age or clinical features to select patients for atrial fibrillation (AF) screening to reduce strokes. We hypothesized that a machine learning (ML) model trained to predict AF risk from 12‑lead electrocardiogram (ECG) would be more efficient than criteria based on clinical variables in indicating a population for AF screening to potentially prevent AF-related stroke. METHODS We retrospectively included all patients with clinical encounters in Geisinger without a prior history of AF. Incidence of AF within 1 year and AF-related strokes within 3 years of the encounter were identified. AF-related stroke was defined as a stroke where AF was diagnosed at the time of stroke or within a year after the stroke. The efficiency of five methods was evaluated for selecting a cohort for AF screening. The methods were selected from four clinical trials (mSToPS, GUARD-AF, SCREEN-AF and STROKESTOP) and the ECG-based ML model. We simulated patient selection for the five methods between the years 2011 and 2014 and evaluated outcomes for 1 year intervals between 2012 and 2015, resulting in a total of twenty 1-year periods. Patients were considered eligible if they met the criteria before the start of the given 1-year period or within that period. The primary outcomes were numbers needed to screen (NNS) for AF and AF-associated stroke. RESULTS The clinical trial models indicated large proportions of the population with a prior ECG for AF screening (up to 31%), coinciding with NNS ranging from 14 to 18 for AF and 249-359 for AF-associated stroke. At comparable sensitivity, the ECG ML model indicated a modest number of patients for screening (14%) and had the highest efficiency in NNS for AF (7.3; up to 60% reduction) and AF-associated stroke (223; up to 38% reduction). CONCLUSIONS An ECG-based ML risk prediction model is more efficient than contemporary AF-screening criteria based on age alone or age and clinical features at indicating a population for AF screening to potentially prevent AF-related strokes.
Collapse
Affiliation(s)
| | - John M Pfeifer
- Geisinger, Danville, PA, USA; Tempus Labs Inc., Chicago, IL, USA
| | | | | | | | | | | | | | | | - Joseph B Leader
- Geisinger, Danville, PA, USA; Tempus Labs Inc., Chicago, IL, USA
| | | | | | - Ruijun Chen
- Geisinger, Danville, PA, USA; Tempus Labs Inc., Chicago, IL, USA
| | | | | | | |
Collapse
|
2
|
Ulloa-Cerna AE, Jing L, Pfeifer JM, Raghunath S, Ruhl JA, Rocha DB, Leader JB, Zimmerman N, Lee G, Steinhubl SR, Good CW, Haggerty CM, Fornwalt BK, Chen R. rECHOmmend: An ECG-based Machine-learning Approach for Identifying Patients at High-risk of Undiagnosed Structural Heart Disease Detectable by Echocardiography. Circulation 2022; 146:36-47. [PMID: 35533093 PMCID: PMC9241668 DOI: 10.1161/circulationaha.121.057869] [Citation(s) in RCA: 19] [Impact Index Per Article: 9.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
Abstract
Background: Timely diagnosis of structural heart disease improves patient outcomes, yet many remain underdiagnosed. While population screening with echocardiography is impractical, ECG-based prediction models can help target high-risk patients. We developed a novel ECG-based machine learning approach to predict multiple structural heart conditions, hypothesizing that a composite model would yield higher prevalence and positive predictive values to facilitate meaningful recommendations for echocardiography. Methods: Using 2 232 130 ECGs linked to electronic health records and echocardiography reports from 484 765 adults between 1984 to 2021, we trained machine learning models to predict the presence or absence of any of 7 echocardiography-confirmed diseases within 1 year. This composite label included the following: moderate or severe valvular disease (aortic/mitral stenosis or regurgitation, tricuspid regurgitation), reduced ejection fraction <50%, or interventricular septal thickness >15 mm. We tested various combinations of input features (demographics, laboratory values, structured ECG data, ECG traces) and evaluated model performance using 5-fold cross-validation, multisite validation trained on 1 site and tested on 10 independent sites, and simulated retrospective deployment trained on pre-2010 data and deployed in 2010. Results: Our composite rECHOmmend model used age, sex, and ECG traces and had a 0.91 area under the receiver operating characteristic curve and a 42% positive predictive value at 90% sensitivity, with a composite label prevalence of 17.9%. Individual disease models had area under the receiver operating characteristic curves from 0.86 to 0.93 and lower positive predictive values from 1% to 31%. Area under the receiver operating characteristic curves for models using different input features ranged from 0.80 to 0.93, increasing with additional features. Multisite validation showed similar results to cross-validation, with an aggregate area under the receiver operating characteristic curve of 0.91 across our independent test set of 10 clinical sites after training on a separate site. Our simulated retrospective deployment showed that for ECGs acquired in patients without preexisting structural heart disease in the year 2010, 11% were classified as high risk and 41% (4.5% of total patients) developed true echocardiography-confirmed disease within 1 year. Conclusions: An ECG-based machine learning model using a composite end point can identify a high-risk population for having undiagnosed, clinically significant structural heart disease while outperforming single-disease models and improving practical utility with higher positive predictive values. This approach can facilitate targeted screening with echocardiography to improve underdiagnosis of structural heart disease.
Collapse
Affiliation(s)
- Alvaro E Ulloa-Cerna
- Department of Translational Data Science and Informatics, Geisinger, Danville, PA
| | - Linyuan Jing
- Department of Translational Data Science and Informatics, Geisinger, Danville, PA
| | - John M Pfeifer
- Department of Translational Data Science and Informatics, Geisinger, Danville, PA; Heart and Vascular Center, Evangelical Hospital, Lewisburg, PA; Tempus Labs Inc, Chicago, IL
| | - Sushravya Raghunath
- Department of Translational Data Science and Informatics, Geisinger, Danville, PA; Tempus Labs Inc, Chicago, IL
| | - Jeffrey A Ruhl
- Department of Translational Data Science and Informatics, Geisinger, Danville, PA
| | - Daniel B Rocha
- Phenomic Analytics and Clinical Data Core, Geisinger, Danville, PA
| | - Joseph B Leader
- Phenomic Analytics and Clinical Data Core, Geisinger, Danville, PA; Tempus Labs Inc, Chicago, IL
| | | | | | - Steven R Steinhubl
- Tempus Labs Inc, Chicago, IL; Scripps Research Translational Institute, La Jolla, CA
| | - Christopher W Good
- Department of Translational Data Science and Informatics, Geisinger, Danville, PA; UPMC Heart and Vascular Institute at UPMC, Hamot, PA
| | - Christopher M Haggerty
- Department of Translational Data Science and Informatics, Geisinger, Danville, PA; Heart Institute, Geisinger, Danville, PA
| | - Brandon K Fornwalt
- Department of Translational Data Science and Informatics, Geisinger, Danville, PA; Tempus Labs Inc, Chicago, IL; Heart Institute, Geisinger, Danville, PA; Department of Radiology, Geisinger, Danville, PA
| | - RuiJun Chen
- Department of Translational Data Science and Informatics, Geisinger, Danville, PA; Tempus Labs Inc, Chicago, IL; Department of Medicine, Geisinger, Danville, PA
| |
Collapse
|
3
|
Fornwalt BK, Pfeifer JM. Promise and Frustration: Machine Learning in Cardiology. Circ Cardiovasc Imaging 2021; 14:e012838. [PMID: 34126753 DOI: 10.1161/circimaging.121.012838] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Affiliation(s)
- Brandon K Fornwalt
- Department of Translational Data Science and Informatics, Geisinger, Danville, PA (B.K.F., J.M.P.).,Department of Radiology and the Heart Institute, Geisinger, Danville, PA (B.K.F.)
| | - John M Pfeifer
- Department of Translational Data Science and Informatics, Geisinger, Danville, PA (B.K.F., J.M.P.).,Heart and Vascular Center, Evangelical Hospital, Lewisburg, PA (J.M.P.)
| |
Collapse
|
4
|
Ulloa Cerna AE, Jing L, Good CW, vanMaanen DP, Raghunath S, Suever JD, Nevius CD, Wehner GJ, Hartzel DN, Leader JB, Alsaid A, Patel AA, Kirchner HL, Pfeifer JM, Carry BJ, Pattichis MS, Haggerty CM, Fornwalt BK. Deep-learning-assisted analysis of echocardiographic videos improves predictions of all-cause mortality. Nat Biomed Eng 2021; 5:546-554. [PMID: 33558735 DOI: 10.1038/s41551-020-00667-9] [Citation(s) in RCA: 20] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2020] [Accepted: 11/24/2020] [Indexed: 01/30/2023]
Abstract
Machine learning promises to assist physicians with predictions of mortality and of other future clinical events by learning complex patterns from historical data, such as longitudinal electronic health records. Here we show that a convolutional neural network trained on raw pixel data in 812,278 echocardiographic videos from 34,362 individuals provides superior predictions of one-year all-cause mortality. The model's predictions outperformed the widely used pooled cohort equations, the Seattle Heart Failure score (measured in an independent dataset of 2,404 patients with heart failure who underwent 3,384 echocardiograms), and a machine learning model involving 58 human-derived variables from echocardiograms and 100 clinical variables derived from electronic health records. We also show that cardiologists assisted by the model substantially improved the sensitivity of their predictions of one-year all-cause mortality by 13% while maintaining prediction specificity. Large unstructured datasets may enable deep learning to improve a wide range of clinical prediction models.
Collapse
Affiliation(s)
- Alvaro E Ulloa Cerna
- Department of Translational Data Science and Informatics, Geisinger, Danville, PA, USA.,Electrical and Computer Engineering Department, University of New Mexico, Albuquerque, NM, USA
| | - Linyuan Jing
- Department of Translational Data Science and Informatics, Geisinger, Danville, PA, USA
| | | | - David P vanMaanen
- Department of Translational Data Science and Informatics, Geisinger, Danville, PA, USA
| | - Sushravya Raghunath
- Department of Translational Data Science and Informatics, Geisinger, Danville, PA, USA
| | - Jonathan D Suever
- Department of Translational Data Science and Informatics, Geisinger, Danville, PA, USA
| | - Christopher D Nevius
- Department of Translational Data Science and Informatics, Geisinger, Danville, PA, USA
| | - Gregory J Wehner
- Department of Biomedical Engineering, University of Kentucky, Lexington, KY, USA
| | - Dustin N Hartzel
- Phenomic Analytics and Clinical Data Core, Geisinger, Danville, PA, USA
| | - Joseph B Leader
- Phenomic Analytics and Clinical Data Core, Geisinger, Danville, PA, USA
| | - Amro Alsaid
- Heart Institute, Geisinger, Danville, PA, USA
| | | | - H Lester Kirchner
- Department of Population Health Sciences, Geisinger, Danville, PA, USA
| | - John M Pfeifer
- Department of Translational Data Science and Informatics, Geisinger, Danville, PA, USA.,Heart and Vascular Center, Evangelical Hospital, Lewisburg, PA, USA
| | | | - Marios S Pattichis
- Electrical and Computer Engineering Department, University of New Mexico, Albuquerque, NM, USA
| | - Christopher M Haggerty
- Department of Translational Data Science and Informatics, Geisinger, Danville, PA, USA.,Heart Institute, Geisinger, Danville, PA, USA
| | - Brandon K Fornwalt
- Department of Translational Data Science and Informatics, Geisinger, Danville, PA, USA. .,Heart Institute, Geisinger, Danville, PA, USA. .,Department of Radiology, Geisinger, Danville, PA, USA.
| |
Collapse
|
5
|
Raghunath S, Pfeifer JM, Ulloa-Cerna AE, Nemani A, Carbonati T, Jing L, vanMaanen DP, Hartzel DN, Ruhl JA, Lagerman BF, Rocha DB, Stoudt NJ, Schneider G, Johnson KW, Zimmerman N, Leader JB, Kirchner HL, Griessenauer CJ, Hafez A, Good CW, Fornwalt BK, Haggerty CM. Deep Neural Networks Can Predict New-Onset Atrial Fibrillation From the 12-Lead ECG and Help Identify Those at Risk of Atrial Fibrillation-Related Stroke. Circulation 2021; 143:1287-1298. [PMID: 33588584 PMCID: PMC7996054 DOI: 10.1161/circulationaha.120.047829] [Citation(s) in RCA: 93] [Impact Index Per Article: 31.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Supplemental Digital Content is available in the text. Atrial fibrillation (AF) is associated with substantial morbidity, especially when it goes undetected. If new-onset AF could be predicted, targeted screening could be used to find it early. We hypothesized that a deep neural network could predict new-onset AF from the resting 12-lead ECG and that this prediction may help identify those at risk of AF-related stroke.
Collapse
Affiliation(s)
- Sushravya Raghunath
- Department of Translational Data Science and Informatics (S.R., A.E.U.-C., L.J., D.P.v.M, J.A.R., N.J.S., G.S., B.K.F., C.M.H.), Geisinger, Danville, PA
| | - John M Pfeifer
- Heart and Vascular Center, Evangelical Hospital, Lewisburg, PA (J.M.P.)
| | - Alvaro E Ulloa-Cerna
- Department of Translational Data Science and Informatics (S.R., A.E.U.-C., L.J., D.P.v.M, J.A.R., N.J.S., G.S., B.K.F., C.M.H.), Geisinger, Danville, PA
| | - Arun Nemani
- Tempus Labs Inc, Chicago, IL (A.N., T.C., K.W.J., N.Z., A.H.)
| | | | - Linyuan Jing
- Department of Translational Data Science and Informatics (S.R., A.E.U.-C., L.J., D.P.v.M, J.A.R., N.J.S., G.S., B.K.F., C.M.H.), Geisinger, Danville, PA
| | - David P vanMaanen
- Department of Translational Data Science and Informatics (S.R., A.E.U.-C., L.J., D.P.v.M, J.A.R., N.J.S., G.S., B.K.F., C.M.H.), Geisinger, Danville, PA
| | - Dustin N Hartzel
- Phenomic Analytics and Clinical Data Core (D.N.H., B.F.L., D.B.R., J.B.L.), Geisinger, Danville, PA
| | - Jeffery A Ruhl
- Department of Translational Data Science and Informatics (S.R., A.E.U.-C., L.J., D.P.v.M, J.A.R., N.J.S., G.S., B.K.F., C.M.H.), Geisinger, Danville, PA
| | - Braxton F Lagerman
- Phenomic Analytics and Clinical Data Core (D.N.H., B.F.L., D.B.R., J.B.L.), Geisinger, Danville, PA
| | - Daniel B Rocha
- Phenomic Analytics and Clinical Data Core (D.N.H., B.F.L., D.B.R., J.B.L.), Geisinger, Danville, PA
| | - Nathan J Stoudt
- Department of Translational Data Science and Informatics (S.R., A.E.U.-C., L.J., D.P.v.M, J.A.R., N.J.S., G.S., B.K.F., C.M.H.), Geisinger, Danville, PA
| | - Gargi Schneider
- Department of Translational Data Science and Informatics (S.R., A.E.U.-C., L.J., D.P.v.M, J.A.R., N.J.S., G.S., B.K.F., C.M.H.), Geisinger, Danville, PA
| | - Kipp W Johnson
- Tempus Labs Inc, Chicago, IL (A.N., T.C., K.W.J., N.Z., A.H.)
| | - Noah Zimmerman
- Tempus Labs Inc, Chicago, IL (A.N., T.C., K.W.J., N.Z., A.H.)
| | - Joseph B Leader
- Phenomic Analytics and Clinical Data Core (D.N.H., B.F.L., D.B.R., J.B.L.), Geisinger, Danville, PA
| | - H Lester Kirchner
- Department of Population Health Sciences (H.L.K.), Geisinger, Danville, PA
| | - Christoph J Griessenauer
- Department of Vascular and Endovascular Neurosurgery (C.J.G.), Geisinger, Danville, PA.,Research Institute of Neurointervention, Paracelsus Medical University, Salzburg, Austria (C.J.G.)
| | - Ashraf Hafez
- Tempus Labs Inc, Chicago, IL (A.N., T.C., K.W.J., N.Z., A.H.)
| | - Christopher W Good
- Heart Institute (C.W.G., B.K.F., C.M.H.), Geisinger, Danville, PA.,Heart and Vascular Institute at University of Pittsburgh Medical Center Hamot, Erie, PA (C.W.G.)
| | - Brandon K Fornwalt
- Department of Translational Data Science and Informatics (S.R., A.E.U.-C., L.J., D.P.v.M, J.A.R., N.J.S., G.S., B.K.F., C.M.H.), Geisinger, Danville, PA.,Heart Institute (C.W.G., B.K.F., C.M.H.), Geisinger, Danville, PA.,Department of Radiology (B.K.F.), Geisinger, Danville, PA
| | - Christopher M Haggerty
- Department of Translational Data Science and Informatics (S.R., A.E.U.-C., L.J., D.P.v.M, J.A.R., N.J.S., G.S., B.K.F., C.M.H.), Geisinger, Danville, PA.,Heart Institute (C.W.G., B.K.F., C.M.H.), Geisinger, Danville, PA
| |
Collapse
|
6
|
Ladapo JA, Pfeifer JM, Pitcavage JM, Williams BA, Choy-Shan AA. Quantifying Sex Differences in Cardiovascular Care Among Patients Evaluated for Suspected Ischemic Heart Disease. J Womens Health (Larchmt) 2018; 28:698-704. [PMID: 30543478 DOI: 10.1089/jwh.2018.7018] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/14/2023] Open
Abstract
Background: Cardiovascular care sex differences are controversial. We examined sex differences in management and clinical outcomes among patients undergoing noninvasive testing for ischemic heart disease (IHD). Methods: In a rural integrated healthcare system, we identified adults age 40-79 without diagnosed IHD who underwent initial evaluation with a cardiac stress test with imaging or coronary computed tomographic angiography (CTA), 2013-2014. We assessed sex differences in statin/aspirin therapy, revascularization, and adverse cardiovascular events. The 2013 American College of Cardiology/American Heart Association statin guidelines and U.S. Preventive Services Task Force aspirin guidelines were applied. Results: Among 2213 patients evaluated for IHD, median age was 57 years, 48.8% were women, and 9% had a positive stress test/CTA. Women were more likely to be missing lipid values than men (p < 0.001). Mean ASCVD risk score at baseline was 7.2% in women versus 12.4% in men (p < 0.001). There was no significant sex difference in statin therapy at baseline or 60-day follow-up. Women were less likely than men to be taking aspirin at baseline (adj. diff. = -8.5%; 95% CI, -4.2 to -12.9) and follow-up (adj. diff. = -7.7%; 95% CI, -3.3 to -12.1). There were no sex differences in revascularization after accounting for obstructive CAD or adverse cardiovascular outcomes during median follow-up of 33 months. Conclusion: In this contemporary cohort of patients with suspected IHD, women were less likely to receive lipid testing and aspirin therapy, but not statin therapy. Women did not experience worse outcomes. Sex differences in statin therapy reported by others may be due to inadequate accounting for baseline risk.
Collapse
Affiliation(s)
- Joseph A Ladapo
- 1 Division of General Internal Medicine and Health Services Research, David Geffen School of Medicine at UCLA, Los Angeles, California
| | | | | | | | - Alana A Choy-Shan
- 3 Department of Medicine, New York University School of Medicine, New York, New York
| |
Collapse
|
7
|
Ladapo JA, Pfeifer JM, Choy-Shan AA, Pitcavage JM, Williams BA. Association of Patient Beliefs and Preferences With Subsequent Testing After Initial Evaluation for Ischemic Heart Disease. JACC Cardiovasc Imaging 2016; 10:1076-1078. [PMID: 28017386 DOI: 10.1016/j.jcmg.2016.09.015] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [What about the content of this article? (0)] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/26/2016] [Revised: 09/07/2016] [Accepted: 09/08/2016] [Indexed: 11/18/2022]
|