1
|
Hadida Barzilai D, Cohen-Shelly M, Sorin V, Zimlichman E, Massalha E, Allison TG, Klang E. Machine learning in cardiac stress test interpretation: a systematic review. EUROPEAN HEART JOURNAL. DIGITAL HEALTH 2024; 5:401-408. [PMID: 39081945 PMCID: PMC11284008 DOI: 10.1093/ehjdh/ztae027] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/24/2023] [Revised: 03/13/2024] [Accepted: 04/01/2024] [Indexed: 08/02/2024]
Abstract
Coronary artery disease (CAD) is a leading health challenge worldwide. Exercise stress testing is a foundational non-invasive diagnostic tool. Nonetheless, its variable accuracy prompts the exploration of more reliable methods. Recent advancements in machine learning (ML), including deep learning and natural language processing, have shown potential in refining the interpretation of stress testing data. Adhering to Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, we conducted a systematic review of ML applications in stress electrocardiogram (ECG) and stress echocardiography for CAD prognosis. Medical Literature Analysis and Retrieval System Online, Web of Science, and the Cochrane Library were used as databases. We analysed the ML models, outcomes, and performance metrics. Overall, seven relevant studies were identified. Machine-learning applications in stress ECGs resulted in sensitivity and specificity improvements. Some models achieved rates of above 96% in both metrics and reduced false positives by up to 21%. In stress echocardiography, ML models demonstrated an increase in diagnostic precision. Some models achieved specificity and sensitivity rates of up to 92.7 and 84.4%, respectively. Natural language processing applications enabled the categorization of stress echocardiography reports, with accuracy rates nearing 98%. Limitations include a small, retrospective study pool and the exclusion of nuclear stress testing, due to its well-documented status. This review indicates the potential of artificial intelligence applications in refining CAD stress testing assessment. Further development for real-world use is warranted.
Collapse
Affiliation(s)
- Dor Hadida Barzilai
- Sami Sagol AI Hub, ARC, Sheba Medical Center, 31 Emek Ha'ela, Ramat Gan 5262000, Israel
| | - Michal Cohen-Shelly
- Sami Sagol AI Hub, ARC, Sheba Medical Center, 31 Emek Ha'ela, Ramat Gan 5262000, Israel
- Leviev Heart Center, Sheba Medical Center, 31 Emek Ha'ela, Ramat Gan 5262000, Israel
| | - Vera Sorin
- School of Medicine, Tel Aviv University, Tel Aviv, Ramat Aviv 69978, Israel
- Department of Diagnostic Radiology, Sheba Medical Center, 31 Emek Ha'ela, Ramat Gan 5262000, Israel
| | - Eyal Zimlichman
- Sami Sagol AI Hub, ARC, Sheba Medical Center, 31 Emek Ha'ela, Ramat Gan 5262000, Israel
- School of Medicine, Tel Aviv University, Tel Aviv, Ramat Aviv 69978, Israel
- Sheba Medical Center, The Sheba Talpiot Medical Leadership Program, 31 Emek Ha'ela, Ramat Gan 5262000, Israel
| | - Eias Massalha
- Leviev Heart Center, Sheba Medical Center, 31 Emek Ha'ela, Ramat Gan 5262000, Israel
| | - Thomas G Allison
- Department of Cardiovascular Medicine, Mayo Clinic, 21 2nd St SW Suite 30, Rochester, MN 55905, USA
- Division of Pediatric Cardiology, Department of Pediatric and Adolescent Medicine, Mayo Clinic, 200 1st St SW, Rochester, MN 55905, USA
| | - Eyal Klang
- Sami Sagol AI Hub, ARC, Sheba Medical Center, 31 Emek Ha'ela, Ramat Gan 5262000, Israel
- School of Medicine, Tel Aviv University, Tel Aviv, Ramat Aviv 69978, Israel
- Department of Diagnostic Radiology, Sheba Medical Center, 31 Emek Ha'ela, Ramat Gan 5262000, Israel
- Sheba Medical Center, The Sheba Talpiot Medical Leadership Program, 31 Emek Ha'ela, Ramat Gan 5262000, Israel
- Division of Data-Driven and Digital Medicine (D3M), Icahn School of Medicine at Mount Sinai, 1 Gustave L. Levy Place, New York, NY 10029-5674, USA
| |
Collapse
|
2
|
Zheng C, Lee MS, Bansal N, Go AS, Chen C, Harrison TN, Fan D, Allen A, Garcia E, Lidgard B, Singer D, An J. Identification of recurrent atrial fibrillation using natural language processing applied to electronic health records. EUROPEAN HEART JOURNAL. QUALITY OF CARE & CLINICAL OUTCOMES 2024; 10:77-88. [PMID: 36997334 PMCID: PMC10785579 DOI: 10.1093/ehjqcco/qcad021] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/10/2023] [Revised: 03/14/2023] [Accepted: 03/29/2023] [Indexed: 04/01/2023]
Abstract
AIMS This study aimed to develop and apply natural language processing (NLP) algorithms to identify recurrent atrial fibrillation (AF) episodes following rhythm control therapy initiation using electronic health records (EHRs). METHODS AND RESULTS We included adults with new-onset AF who initiated rhythm control therapies (ablation, cardioversion, or antiarrhythmic medication) within two US integrated healthcare delivery systems. A code-based algorithm identified potential AF recurrence using diagnosis and procedure codes. An automated NLP algorithm was developed and validated to capture AF recurrence from electrocardiograms, cardiac monitor reports, and clinical notes. Compared with the reference standard cases confirmed by physicians' adjudication, the F-scores, sensitivity, and specificity were all above 0.90 for the NLP algorithms at both sites. We applied the NLP and code-based algorithms to patients with incident AF (n = 22 970) during the 12 months after initiating rhythm control therapy. Applying the NLP algorithms, the percentages of patients with AF recurrence for sites 1 and 2 were 60.7% and 69.9% (ablation), 64.5% and 73.7% (cardioversion), and 49.6% and 55.5% (antiarrhythmic medication), respectively. In comparison, the percentages of patients with code-identified AF recurrence for sites 1 and 2 were 20.2% and 23.7% for ablation, 25.6% and 28.4% for cardioversion, and 20.0% and 27.5% for antiarrhythmic medication, respectively. CONCLUSION When compared with a code-based approach alone, this study's high-performing automated NLP method identified significantly more patients with recurrent AF. The NLP algorithms could enable efficient evaluation of treatment effectiveness of AF therapies in large populations and help develop tailored interventions.
Collapse
Affiliation(s)
- Chengyi Zheng
- Research and Evaluation Department, Kaiser Permanente Southern California,100 S Los Robles Ave, 2nd Floor, Pasadena, CA 91101, USA
| | - Ming-sum Lee
- Department of Cardiology, Kaiser Permanente Los Angeles Medical Center, Los Angeles, CA 90027, USA
| | - Nisha Bansal
- Kidney Research Institute, Division of Nephrology, University of Washington, Seattle, WA 98104, USA
| | - Alan S Go
- Division of Research, Kaiser Permanente Northern California, Oakland, CA 94612, USA
- Department of Health Systems Science, Kaiser Permanente Bernard J. Tyson School of Medicine, Pasadena, CA 91101, USA
- Department of Medicine and Department of Epidemiology and Biostatistics, University of California, San Francisco, San Francisco, CA 94158, USA
- Departments of Medicine, Stanford University, Palo Alto, CA 94305, USA
| | - Cheng Chen
- Department of Cardiology, Kaiser Permanente Fontana Medical Center, Fontana, CA 92335, USA
| | - Teresa N Harrison
- Research and Evaluation Department, Kaiser Permanente Southern California,100 S Los Robles Ave, 2nd Floor, Pasadena, CA 91101, USA
| | - Dongjie Fan
- Division of Research, Kaiser Permanente Northern California, Oakland, CA 94612, USA
| | - Amanda Allen
- Division of Research, Kaiser Permanente Northern California, Oakland, CA 94612, USA
| | - Elisha Garcia
- Division of Research, Kaiser Permanente Northern California, Oakland, CA 94612, USA
| | - Ben Lidgard
- Department of Health Systems Science, Kaiser Permanente Bernard J. Tyson School of Medicine, Pasadena, CA 91101, USA
| | - Daniel Singer
- Clinical Epidemiology Unit, Massachusetts General Hospital, Boston, MA 02114, USA
| | - Jaejin An
- Research and Evaluation Department, Kaiser Permanente Southern California,100 S Los Robles Ave, 2nd Floor, Pasadena, CA 91101, USA
- Department of Health Systems Science, Kaiser Permanente Bernard J. Tyson School of Medicine, Pasadena, CA 91101, USA
| |
Collapse
|
3
|
Lu J, Stewart J, Bennamoun M, Goudie A, Eshraghian J, Ihdayhid A, Sanfilippo F, Small GR, Chow BJ, Dwivedi G. Deep learning model to predict exercise stress test results: Optimizing the diagnostic test selection strategy and reduce wastage in suspected coronary artery disease patients. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 240:107717. [PMID: 37454499 DOI: 10.1016/j.cmpb.2023.107717] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/30/2022] [Revised: 05/27/2023] [Accepted: 07/08/2023] [Indexed: 07/18/2023]
Abstract
BACKGROUND Cardiac exercise stress testing (EST) offers a non-invasive way in the management of patients with suspected coronary artery disease (CAD). However, up to 30% EST results are either inconclusive or non-diagnostic, which results in significant resource wastage. Our aim was to build machine learning (ML) based models, using patients demographic (age, sex) and pre-test clinical information (reason for performing test, medications, blood pressure, heart rate, and resting electrocardiogram), capable of predicting EST results beforehand including those with inconclusive or non-diagnostic results. METHODS A total of 30,710 patients (mean age 54.0 years, 69% male) were included in the study with 25% randomly sampled in the test set, and the remaining samples were split into a train and validation set with a ratio of 9:1. We constructed different ML models from pre-test variables and compared their discriminant power using the area under the receiver operating characteristic curve (AUC). RESULTS A network of Oblivious Decision Trees provided the best discriminant power (AUC=0.83, sensitivity=69%, specificity=0.78%) for predicting inconclusive EST results. A total of 2010 inconclusive ESTs were correctly identified in the testing set. CONCLUSIONS Our ML model, developed using demographic and pre-test clinical information, can accurately predict EST results and could be used to identify patients with inconclusive or non-diagnostic results beforehand. Our system could thus be used as a personalised decision support tool by clinicians for optimizing the diagnostic test selection strategy for CAD patients and to reduce healthcare expenditure by reducing nondiagnostic or inconclusive ESTs.
Collapse
Affiliation(s)
- Juan Lu
- Department of Computer Science and Software Engineering, The University of Western Australia, Australia; Medical School, Faculty of Health and Medical Sciences, University of Western Australia, Perth, Australia; Harry Perkins Institute of Medical Research, Perth, Australia
| | - Jonathon Stewart
- Medical School, Faculty of Health and Medical Sciences, University of Western Australia, Perth, Australia; Harry Perkins Institute of Medical Research, Perth, Australia
| | - Mohammed Bennamoun
- Department of Computer Science and Software Engineering, The University of Western Australia, Australia
| | - Adrian Goudie
- Emergency Department, Fiona Stanley Hospital, Perth, Australia
| | - Jason Eshraghian
- Department of Computer Science and Software Engineering, The University of Western Australia, Australia; Medical School, Faculty of Health and Medical Sciences, University of Western Australia, Perth, Australia
| | - Abdul Ihdayhid
- Medical School, Faculty of Health and Medical Sciences, University of Western Australia, Perth, Australia; Harry Perkins Institute of Medical Research, Perth, Australia; Cardiology Department, Fiona Stanley Hospital, Perth, Australia
| | - Frank Sanfilippo
- School of Population and Global Health, The University of Western Australia, Australia
| | - Gary R Small
- Division of Cardiology, University of Ottawa Heart Institute, Department of Medicine, University of Ottawa, Canada
| | - Benjamin Jw Chow
- Division of Cardiology, University of Ottawa Heart Institute, Department of Medicine, University of Ottawa, Canada
| | - Girish Dwivedi
- Medical School, Faculty of Health and Medical Sciences, University of Western Australia, Perth, Australia; Harry Perkins Institute of Medical Research, Perth, Australia; Cardiology Department, Fiona Stanley Hospital, Perth, Australia.
| |
Collapse
|
4
|
Vilela EM, Oliveira C, Oliveira C, Torres S, Sampaio F, Primo J, Ribeiro J, Teixeira M, Oliveira M, Bettencourt N, Viamonte S, Fontes-Carvalho R. Sixty years of the Bruce protocol: reappraising the contemporary role of exercise stress testing with electrocardiographic monitoring. Porto Biomed J 2023; 8:e235. [PMID: 37846299 PMCID: PMC10575366 DOI: 10.1097/j.pbj.0000000000000235] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/02/2023] [Accepted: 09/18/2023] [Indexed: 10/18/2023] Open
Abstract
The cardiovascular response to exercise has long been a focus of interest. Over a century ago, the first descriptions of electrocardiographic changes occurring during exercise highlighted the possible relevance of this dynamic assessment. In this background, the inception of the Bruce protocol circa 60 years ago allowed for a major leap in this field by providing a standardized framework with which to address this issue, by means of an integrated and structured methodology. Since then, exercise stress testing with electrocardiographic monitoring (ExECG) has become one of the most widely appraised tests in cardiovascular medicine. Notably, past few decades have been profoundly marked by substantial advances in the approach to cardiovascular disease, challenging prior notions concerning both its physiopathology and overall management. Among these, the ever-evolving presentations of cardiovascular disease coupled with the development and implementation of several novel diagnostic modalities (both invasive and noninvasive) has led to a shifting paradigm in the application of ExECG. This technique, however, has continuously shown to be of added value across various momentums of the cardiovascular continuum, as depicted in several contemporary guidelines. This review provides a pragmatical reflexion on the development of ExECG, presenting a comprehensive overview concerning the current role of this modality, its challenges, and its future perspectives.
Collapse
Affiliation(s)
- Eduardo M. Vilela
- Cardiology Department, Centro Hospitalar de Vila Nova de Gaia/Espinho, Vila Nova de Gaia, Portugal
| | - Cátia Oliveira
- Cardiology Department, Hospital de Braga, Braga, Portugal
- Faculty of Medicine, Minho University, Braga, Portugal
| | - Cláudia Oliveira
- Cardiology Department, Centro Hospitalar de Vila Nova de Gaia/Espinho, Vila Nova de Gaia, Portugal
| | - Susana Torres
- Cardiology Department, Centro Hospitalar de Vila Nova de Gaia/Espinho, Vila Nova de Gaia, Portugal
| | - Francisco Sampaio
- Cardiology Department, Centro Hospitalar de Vila Nova de Gaia/Espinho, Vila Nova de Gaia, Portugal
- Cardiovascular Research Center (UniC@RISE), Faculty of Medicine, University of Porto, Porto, Portugal
| | - João Primo
- Cardiology Department, Centro Hospitalar de Vila Nova de Gaia/Espinho, Vila Nova de Gaia, Portugal
| | - José Ribeiro
- Cardiology Department, Centro Hospitalar de Vila Nova de Gaia/Espinho, Vila Nova de Gaia, Portugal
| | - Madalena Teixeira
- Cardiology Department, Centro Hospitalar de Vila Nova de Gaia/Espinho, Vila Nova de Gaia, Portugal
| | - Marco Oliveira
- Cardiology Department, Centro Hospitalar de Vila Nova de Gaia/Espinho, Vila Nova de Gaia, Portugal
| | - Nuno Bettencourt
- Cardiovascular Research Center (UniC@RISE), Faculty of Medicine, University of Porto, Porto, Portugal
| | - Sofia Viamonte
- North Rehabilitation Centre, Centro Hospitalar de Vila Nova de Gaia/Espinho, Vila Nova de Gaia, Portugal
| | - Ricardo Fontes-Carvalho
- Cardiology Department, Centro Hospitalar de Vila Nova de Gaia/Espinho, Vila Nova de Gaia, Portugal
- Cardiovascular Research Center (UniC@RISE), Faculty of Medicine, University of Porto, Porto, Portugal
| |
Collapse
|
5
|
Broyles D, Philibert R. Precision epigenetics provides a scalable pathway for improving coronary heart disease care globally. Epigenomics 2023; 15:805-818. [PMID: 37702023 DOI: 10.2217/epi-2023-0233] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/14/2023] Open
Abstract
Coronary heart disease (CHD) is the world's leading cause of death. Up to 90% of all CHD deaths are preventable, but effective prevention of this mortality requires more scalable, precise methods for assessing CHD status and monitoring treatment response. Unfortunately, current diagnostic methods have barriers to implementation, particularly in rural areas and lower-income countries. This gap may be bridged by highly scalable advances in DNA methylation testing methods and artificial intelligence. Herein, we review prior studies of CHD related to methylation alone and in combination with other biovariables. We compare these new methods with established methods for diagnosing CHD. Finally, we outline pathways through which methylation-based testing methods may allow the democratization of improved methods for assessing CHD globally.
Collapse
Affiliation(s)
- Damon Broyles
- Mercy Technology Services, St. Louis, MO 63127, USA
- Mercy Precision Medicine, Chesterfield, MO 63017, USA
| | - Robert Philibert
- Department of Psychiatry, University of Iowa, Iowa City, IA 52242, USA
- Cardio Diagnostics Inc, Chicago, IL 60642, USA
- Department of Biomedical Engineering, University of Iowa, Iowa City, IA 52242, USA
| |
Collapse
|
6
|
Dewaswala N, Chen D, Bhopalwala H, Kaggal VC, Murphy SP, Bos JM, Geske JB, Gersh BJ, Ommen SR, Araoz PA, Ackerman MJ, Arruda-Olson AM. Natural language processing for identification of hypertrophic cardiomyopathy patients from cardiac magnetic resonance reports. BMC Med Inform Decis Mak 2022; 22:272. [PMID: 36258218 PMCID: PMC9580188 DOI: 10.1186/s12911-022-02017-y] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/24/2020] [Accepted: 10/10/2022] [Indexed: 11/30/2022] Open
Abstract
Background Cardiac magnetic resonance (CMR) imaging is important for diagnosis and risk stratification of hypertrophic cardiomyopathy (HCM) patients. However, collection of information from large numbers of CMR reports by manual review is time-consuming, error-prone and costly. Natural language processing (NLP) is an artificial intelligence method for automated extraction of information from narrative text including text in CMR reports in electronic health records (EHR). Our objective was to assess whether NLP can accurately extract diagnosis of HCM from CMR reports.
Methods An NLP system with two tiers was developed for information extraction from narrative text in CMR reports; the first tier extracted information regarding HCM diagnosis while the second extracted categorical and numeric concepts for HCM classification. We randomly allocated 200 HCM patients with CMR reports from 2004 to 2018 into training (100 patients with 185 CMR reports) and testing sets (100 patients with 206 reports). Results NLP algorithms demonstrated very high performance compared to manual annotation. The algorithm to extract HCM diagnosis had accuracy of 0.99. The accuracy for categorical concepts included HCM morphologic subtype 0.99, systolic anterior motion of the mitral valve 0.96, mitral regurgitation 0.93, left ventricular (LV) obstruction 0.94, location of obstruction 0.92, apical pouch 0.98, LV delayed enhancement 0.93, left atrial enlargement 0.99 and right atrial enlargement 0.98. Accuracy for numeric concepts included maximal LV wall thickness 0.96, LV mass 0.99, LV mass index 0.98, LV ejection fraction 0.98 and right ventricular ejection fraction 0.99. Conclusions NLP identified and classified HCM from CMR narrative text reports with very high performance.
Supplementary Information The online version contains supplementary material available at 10.1186/s12911-022-02017-y.
Collapse
Affiliation(s)
- Nakeya Dewaswala
- Department of Cardiovascular Medicine, Mayo Clinic Rochester, Rochester, MN, USA
| | - David Chen
- Department of Cardiovascular Surgery, Cleveland Clinic, OH, Cleveland, USA
| | - Huzefa Bhopalwala
- Department of Cardiovascular Medicine, Mayo Clinic Rochester, Rochester, MN, USA
| | - Vinod C Kaggal
- Enterprise Technology Services, Shared Service Offices, Mayo Clinic, MN, Rochester, USA
| | - Sean P Murphy
- Advanced Analytics Services, Mayo Clinic Rochester, Rochester, MN, USA
| | - J Martijn Bos
- Department of Cardiovascular Medicine, Mayo Clinic Rochester, Rochester, MN, USA
| | - Jeffrey B Geske
- Department of Cardiovascular Medicine, Mayo Clinic Rochester, Rochester, MN, USA
| | - Bernard J Gersh
- Department of Cardiovascular Medicine, Mayo Clinic Rochester, Rochester, MN, USA
| | - Steve R Ommen
- Department of Cardiovascular Medicine, Mayo Clinic Rochester, Rochester, MN, USA
| | - Philip A Araoz
- Department of Radiology, Mayo Clinic Rochester, Rochester, MN, USA
| | - Michael J Ackerman
- Department of Cardiovascular Medicine, Mayo Clinic Rochester, Rochester, MN, USA.,Department of Pediatric and Adolescent Medicine, Mayo Clinic Rochester, Rochester, MN, USA.,Department of Molecular Pharmacology and Experimental Therapeutics, Mayo Clinic Rochester, Rochester, MN, USA
| | | |
Collapse
|
7
|
Zheng C, Sun BC, Wu YL, Ferencik M, Lee MS, Redberg RF, Kawatkar AA, Musigdilok VV, Sharp AL. Automated interpretation of stress echocardiography reports using natural language processing. EUROPEAN HEART JOURNAL. DIGITAL HEALTH 2022; 3:626-637. [PMID: 36710893 PMCID: PMC9779789 DOI: 10.1093/ehjdh/ztac047] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/31/2022] [Revised: 08/08/2022] [Indexed: 02/01/2023]
Abstract
Aims Stress echocardiography (SE) findings and interpretations are commonly documented in free-text reports. Reusing SE results requires laborious manual reviews. This study aimed to develop and validate an automated method for abstracting SE reports in a large cohort. Methods and results This study included adult patients who had SE within 30 days of their emergency department visit for suspected acute coronary syndrome in a large integrated healthcare system. An automated natural language processing (NLP) algorithm was developed to abstract SE reports and classify overall SE results into normal, non-diagnostic, infarction, and ischaemia categories. Randomly selected reports (n = 140) were double-blindly reviewed by cardiologists to perform criterion validity of the NLP algorithm. Construct validity was tested on the entire cohort using abstracted SE data and additional clinical variables. The NLP algorithm abstracted 6346 consecutive SE reports. Cardiologists had good agreements on the overall SE results on the 140 reports: Kappa (0.83) and intraclass correlation coefficient (0.89). The NLP algorithm achieved 98.6% specificity and negative predictive value, 95.7% sensitivity, positive predictive value, and F-score on the overall SE results and near-perfect scores on ischaemia findings. The 30-day acute myocardial infarction or death outcomes were highest among patients with ischaemia (5.0%), followed by infarction (1.4%), non-diagnostic (0.8%), and normal (0.3%) results. We found substantial variations in the format and quality of SE reports, even within the same institution. Conclusions Natural language processing is an accurate and efficient method for abstracting unstructured SE reports. This approach creates new opportunities for research, public health measures, and care improvement.
Collapse
Affiliation(s)
- Chengyi Zheng
- Corresponding author. Tel: 1-626-376-7029, Fax: 626-564-3694,
| | - Benjamin C Sun
- Department of Emergency Medicine and Leonard Davis Institute, University of Pennsylvania, Philadelphia, PA 19104, USA
| | - Yi-Lin Wu
- Research and Evaluation Department, Kaiser Permanente Southern California, 100 S Los Robles Ave, 2nd Floor, Pasadena, CA 91101, USA
| | - Maros Ferencik
- Oregon Health and Science University, Knight Cardiovascular Institute, Portland, OR 97239, USA
| | - Ming-Sum Lee
- Division of Cardiology, Kaiser Permanente Southern California, Los Angeles Medical Center, Los Angeles, CA 90027, USA
| | - Rita F Redberg
- Division of Cardiology, University of California, San Francisco, CA 94143, USA
| | - Aniket A Kawatkar
- Research and Evaluation Department, Kaiser Permanente Southern California, 100 S Los Robles Ave, 2nd Floor, Pasadena, CA 91101, USA
| | - Visanee V Musigdilok
- Research and Evaluation Department, Kaiser Permanente Southern California, 100 S Los Robles Ave, 2nd Floor, Pasadena, CA 91101, USA
| | - Adam L Sharp
- Research and Evaluation Department, Kaiser Permanente Southern California, 100 S Los Robles Ave, 2nd Floor, Pasadena, CA 91101, USA,Clinical Science Department, Kaiser Permanente Bernard J. Tyson School of Medicine, Pasadena, CA 91101, USA
| |
Collapse
|
8
|
Zheng C, Sun BC, Wu YL, Ferencik M, Lee MS, Redberg RF, Kawatkar AA, Musigdilok VV, Sharp AL. Automated abstraction of myocardial perfusion imaging reports using natural language processing. J Nucl Cardiol 2022; 29:1178-1187. [PMID: 33155169 PMCID: PMC8096860 DOI: 10.1007/s12350-020-02401-z] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/09/2020] [Accepted: 09/29/2020] [Indexed: 01/11/2023]
Abstract
BACKGROUND Findings and interpretations of myocardial perfusion imaging (MPI) studies are documented in free-text MPI reports. MPI results are essential for research, but manual review is prohibitively time consuming. This study aimed to develop and validate an automated method to abstract MPI reports. METHODS We developed a natural language processing (NLP) algorithm to abstract MPI reports. Randomly selected reports were double-blindly reviewed by two cardiologists to validate the NLP algorithm. Secondary analyses were performed to describe patient outcomes based on abstracted-MPI results on 16,957 MPI tests from adult patients evaluated for suspected ACS. RESULTS The NLP algorithm achieved high sensitivity (96.7%) and specificity (98.9%) on the MPI categorical results and had a similar degree of agreement compared to the physician reviewers. Patients with abnormal MPI results had higher rates of 30-day acute myocardial infarction or death compared to patients with normal results. We identified issues related to the quality of the reports that not only affect communication with referring physicians but also challenges for automated abstraction. CONCLUSION NLP is an accurate and efficient strategy to abstract results from the free-text MPI reports. Our findings will facilitate future research to understand the benefits of MPI studies but requires validation in other settings.
Collapse
Affiliation(s)
- Chengyi Zheng
- Research and Evaluation Department, Kaiser Permanente Southern California, 100 S Los Robles Ave, 2nd Floor, Pasadena, CA, 91101, USA.
| | - Benjamin C Sun
- Department of Emergency Medicine and Leonard Davis Institute, University of Pennsylvania, Philadelphia, PA, USA
| | - Yi-Lin Wu
- Research and Evaluation Department, Kaiser Permanente Southern California, 100 S Los Robles Ave, 2nd Floor, Pasadena, CA, 91101, USA
| | - Maros Ferencik
- Knight Cardiovascular Institute, Oregon Health and Science University, Portland, OR, USA
| | - Ming-Sum Lee
- Division of Cardiology, Kaiser Permanente Southern California, Los Angeles Medical Center, Los Angeles, CA, USA
| | - Rita F Redberg
- Division of Cardiology, University of California, San Francisco, San Francisco, CA, USA
| | - Aniket A Kawatkar
- Research and Evaluation Department, Kaiser Permanente Southern California, 100 S Los Robles Ave, 2nd Floor, Pasadena, CA, 91101, USA
| | - Visanee V Musigdilok
- Research and Evaluation Department, Kaiser Permanente Southern California, 100 S Los Robles Ave, 2nd Floor, Pasadena, CA, 91101, USA
| | - Adam L Sharp
- Research and Evaluation Department, Kaiser Permanente Southern California, 100 S Los Robles Ave, 2nd Floor, Pasadena, CA, 91101, USA
| |
Collapse
|
9
|
Reading Turchioe M, Volodarskiy A, Pathak J, Wright DN, Tcheng JE, Slotwiner D. Systematic review of current natural language processing methods and applications in cardiology. Heart 2021; 108:909-916. [PMID: 34711662 DOI: 10.1136/heartjnl-2021-319769] [Citation(s) in RCA: 29] [Impact Index Per Article: 9.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/28/2021] [Accepted: 09/29/2021] [Indexed: 01/16/2023] Open
Abstract
Natural language processing (NLP) is a set of automated methods to organise and evaluate the information contained in unstructured clinical notes, which are a rich source of real-world data from clinical care that may be used to improve outcomes and understanding of disease in cardiology. The purpose of this systematic review is to provide an understanding of NLP, review how it has been used to date within cardiology and illustrate the opportunities that this approach provides for both research and clinical care. We systematically searched six scholarly databases (ACM Digital Library, Arxiv, Embase, IEEE Explore, PubMed and Scopus) for studies published in 2015-2020 describing the development or application of NLP methods for clinical text focused on cardiac disease. Studies not published in English, lacking a description of NLP methods, non-cardiac focused and duplicates were excluded. Two independent reviewers extracted general study information, clinical details and NLP details and appraised quality using a checklist of quality indicators for NLP studies. We identified 37 studies developing and applying NLP in heart failure, imaging, coronary artery disease, electrophysiology, general cardiology and valvular heart disease. Most studies used NLP to identify patients with a specific diagnosis and extract disease severity using rule-based NLP methods. Some used NLP algorithms to predict clinical outcomes. A major limitation is the inability to aggregate findings across studies due to vastly different NLP methods, evaluation and reporting. This review reveals numerous opportunities for future NLP work in cardiology with more diverse patient samples, cardiac diseases, datasets, methods and applications.
Collapse
Affiliation(s)
- Meghan Reading Turchioe
- Department of Population Health Sciences, Division of Health Informatics, Weill Cornell Medicine, New York, New York, USA
| | - Alexander Volodarskiy
- Department of Medicine, Division of Cardiology, NewYork-Presbyterian Hospital, New York, New York, USA
| | - Jyotishman Pathak
- Department of Population Health Sciences, Division of Health Informatics, Weill Cornell Medicine, New York, New York, USA
| | - Drew N Wright
- Samuel J. Wood Library & C.V. Starr Biomedical Information Center, Weill Cornell Medical College, New York, New York, USA
| | - James Enlou Tcheng
- Department of Medicine, Duke University School of Medicine, Durham, North Carolina, USA
| | - David Slotwiner
- Department of Population Health Sciences, Division of Health Informatics, Weill Cornell Medicine, New York, New York, USA.,Department of Medicine, Division of Cardiology, NewYork-Presbyterian Hospital, New York, New York, USA
| |
Collapse
|
10
|
Zheng C, Duffy J, Liu ILA, Sy LS, Navarro RA, Kim SS, Ryan DS, Chen W, Qian L, Mercado C, Jacobsen SJ. Identifying Cases of Shoulder Injury Related to Vaccine Administration (SIRVA) in the United States: Development and Validation of a Natural Language Processing Method (Preprint). JMIR Public Health Surveill 2021; 8:e30426. [PMID: 35608886 PMCID: PMC9175103 DOI: 10.2196/30426] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2021] [Revised: 02/22/2022] [Accepted: 04/26/2022] [Indexed: 11/13/2022] Open
Abstract
Background Shoulder injury related to vaccine administration (SIRVA) accounts for more than half of all claims received by the National Vaccine Injury Compensation Program. However, due to the difficulty of finding SIRVA cases in large health care databases, population-based studies are scarce. Objective The goal of the research was to develop a natural language processing (NLP) method to identify SIRVA cases from clinical notes. Methods We conducted the study among members of a large integrated health care organization who were vaccinated between April 1, 2016, and December 31, 2017, and had subsequent diagnosis codes indicative of shoulder injury. Based on a training data set with a chart review reference standard of 164 cases, we developed an NLP algorithm to extract shoulder disorder information, including prior vaccination, anatomic location, temporality and causality. The algorithm identified 3 groups of positive SIRVA cases (definite, probable, and possible) based on the strength of evidence. We compared NLP results to a chart review reference standard of 100 vaccinated cases. We then applied the final automated NLP algorithm to a broader cohort of vaccinated persons with a shoulder injury diagnosis code and performed manual chart confirmation on a random sample of NLP-identified definite cases and all NLP-identified probable and possible cases. Results In the validation sample, the NLP algorithm had 100% accuracy for identifying 4 SIRVA cases and 96 cases without SIRVA. In the broader cohort of 53,585 vaccinations, the NLP algorithm identified 291 definite, 124 probable, and 52 possible SIRVA cases. The chart-confirmation rates for these groups were 95.5% (278/291), 67.7% (84/124), and 17.3% (9/52), respectively. Conclusions The algorithm performed with high sensitivity and reasonable specificity in identifying positive SIRVA cases. The NLP algorithm can potentially be used in future population-based studies to identify this rare adverse event, avoiding labor-intensive chart review validation.
Collapse
Affiliation(s)
- Chengyi Zheng
- Department of Research and Evaluation, Kaiser Permanente Southern California, Pasadena, CA, United States
| | - Jonathan Duffy
- Immunization Safety Office, Centers for Disease Control and Prevention, Atlanta, GA, United States
| | - In-Lu Amy Liu
- Department of Research and Evaluation, Kaiser Permanente Southern California, Pasadena, CA, United States
| | - Lina S Sy
- Department of Research and Evaluation, Kaiser Permanente Southern California, Pasadena, CA, United States
| | - Ronald A Navarro
- Kaiser Permanente South Bay Medical Center, Harbor City, CA, United States
| | - Sunhea S Kim
- Department of Research and Evaluation, Kaiser Permanente Southern California, Pasadena, CA, United States
| | - Denison S Ryan
- Department of Research and Evaluation, Kaiser Permanente Southern California, Pasadena, CA, United States
| | - Wansu Chen
- Department of Research and Evaluation, Kaiser Permanente Southern California, Pasadena, CA, United States
| | - Lei Qian
- Department of Research and Evaluation, Kaiser Permanente Southern California, Pasadena, CA, United States
| | - Cheryl Mercado
- Department of Research and Evaluation, Kaiser Permanente Southern California, Pasadena, CA, United States
| | - Steven J Jacobsen
- Department of Research and Evaluation, Kaiser Permanente Southern California, Pasadena, CA, United States
| |
Collapse
|
11
|
Zheng C, Sun BC, Wu YL, Lee MS, Shen E, Redberg RF, Ferencik M, Natsui S, Kawatkar AA, Musigdilok VV, Sharp AL. Automated Identification and Extraction of Exercise Treadmill Test Results. J Am Heart Assoc 2020; 9:e014940. [PMID: 32079480 PMCID: PMC7335560 DOI: 10.1161/jaha.119.014940] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/27/2022]
Abstract
Background Noninvasive cardiac tests, including exercise treadmill tests (ETTs), are commonly utilized in the evaluation of patients in the emergency department with suspected acute coronary syndrome. However, there are ongoing debates on their clinical utility and cost‐effectiveness. It is important to be able to use ETT results for research, but manual review is prohibitively time‐consuming for large studies. We developed and validated an automated method to interpret ETT results from electronic health records. To demonstrate the algorithm's utility, we tested the associations between ETT results with 30‐day patient outcomes in a large population. Methods and Results A retrospective analysis of adult emergency department encounters resulting in an ETT within 30 days was performed. A set of randomly selected reports were double‐blind reviewed by 2 physicians to validate a natural language processing algorithm designed to categorize ETT results into normal, ischemic, nondiagnostic, and equivocal categories. Natural language processing then searched and categorized results of 5214 ETT reports. The natural language processing algorithm achieved 96.4% sensitivity and 94.8% specificity in identifying normal versus all other categories. The rates of 30‐day death or acute myocardial infarction varied (P<0.001) by categories for normal (0.08%), ischemic (1.9%), nondiagnostic (0.77%), and equivocal (0.58%) groups achieving good discrimination (C‐statistic, 0.81; 95% CI, 0.7–0.92). Conclusions Natural language processing is an accurate and efficient strategy to facilitate large‐scale outcome studies of noninvasive cardiac tests. We found that most patients are at low risk and have normal ETT results, while those with abnormal, nondiagnostic, or equivocal results have slightly higher risks and warrant future investigation.
Collapse
Affiliation(s)
- Chengyi Zheng
- Research and Evaluation Department Kaiser Permanente Southern California Pasadena CA
| | - Benjamin C Sun
- Department of Emergency Medicine University of Pennsylvania Philadelphia PA
| | - Yi-Lin Wu
- Research and Evaluation Department Kaiser Permanente Southern California Pasadena CA
| | - Ming-Sum Lee
- Division of Cardiology Kaiser Permanente Southern California, Los Angeles Medical Center Los Angeles CA
| | - Ernest Shen
- Research and Evaluation Department Kaiser Permanente Southern California Pasadena CA
| | - Rita F Redberg
- Division of Cardiology University of California, San Francisco San Francisco CA
| | - Maros Ferencik
- Knight Cardiovascular Institute Oregon Health and Science University Portland OR
| | - Shaw Natsui
- National Clinician Scholars Program Department of Emergency Medicine University of California, Los Angeles Los Angeles CA
| | - Aniket A Kawatkar
- Research and Evaluation Department Kaiser Permanente Southern California Pasadena CA
| | - Visanee V Musigdilok
- Research and Evaluation Department Kaiser Permanente Southern California Pasadena CA
| | - Adam L Sharp
- Research and Evaluation Department Kaiser Permanente Southern California Pasadena CA
| |
Collapse
|