Automated Identification and Extraction of Exercise Treadmill Test Results

Machine learning in cardiac stress test interpretation: a systematic review

Coronary artery disease (CAD) is a leading health challenge worldwide. Exercise stress testing is a foundational non-invasive diagnostic tool. Nonetheless, its variable accuracy prompts the exploration of more reliable methods. Recent advancements in machine learning (ML), including deep learning and natural language processing, have shown potential in refining the interpretation of stress testing data. Adhering to Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, we conducted a systematic review of ML applications in stress electrocardiogram (ECG) and stress echocardiography for CAD prognosis. Medical Literature Analysis and Retrieval System Online, Web of Science, and the Cochrane Library were used as databases. We analysed the ML models, outcomes, and performance metrics. Overall, seven relevant studies were identified. Machine-learning applications in stress ECGs resulted in sensitivity and specificity improvements. Some models achieved rates of above 96% in both metrics and reduced false positives by up to 21%. In stress echocardiography, ML models demonstrated an increase in diagnostic precision. Some models achieved specificity and sensitivity rates of up to 92.7 and 84.4%, respectively. Natural language processing applications enabled the categorization of stress echocardiography reports, with accuracy rates nearing 98%. Limitations include a small, retrospective study pool and the exclusion of nuclear stress testing, due to its well-documented status. This review indicates the potential of artificial intelligence applications in refining CAD stress testing assessment. Further development for real-world use is warranted.

Identification of recurrent atrial fibrillation using natural language processing applied to electronic health records

AIMS

This study aimed to develop and apply natural language processing (NLP) algorithms to identify recurrent atrial fibrillation (AF) episodes following rhythm control therapy initiation using electronic health records (EHRs).

METHODS AND RESULTS

We included adults with new-onset AF who initiated rhythm control therapies (ablation, cardioversion, or antiarrhythmic medication) within two US integrated healthcare delivery systems. A code-based algorithm identified potential AF recurrence using diagnosis and procedure codes. An automated NLP algorithm was developed and validated to capture AF recurrence from electrocardiograms, cardiac monitor reports, and clinical notes. Compared with the reference standard cases confirmed by physicians' adjudication, the F-scores, sensitivity, and specificity were all above 0.90 for the NLP algorithms at both sites. We applied the NLP and code-based algorithms to patients with incident AF (n = 22 970) during the 12 months after initiating rhythm control therapy. Applying the NLP algorithms, the percentages of patients with AF recurrence for sites 1 and 2 were 60.7% and 69.9% (ablation), 64.5% and 73.7% (cardioversion), and 49.6% and 55.5% (antiarrhythmic medication), respectively. In comparison, the percentages of patients with code-identified AF recurrence for sites 1 and 2 were 20.2% and 23.7% for ablation, 25.6% and 28.4% for cardioversion, and 20.0% and 27.5% for antiarrhythmic medication, respectively.

CONCLUSION

When compared with a code-based approach alone, this study's high-performing automated NLP method identified significantly more patients with recurrent AF. The NLP algorithms could enable efficient evaluation of treatment effectiveness of AF therapies in large populations and help develop tailored interventions.

Deep learning model to predict exercise stress test results: Optimizing the diagnostic test selection strategy and reduce wastage in suspected coronary artery disease patients

BACKGROUND

Cardiac exercise stress testing (EST) offers a non-invasive way in the management of patients with suspected coronary artery disease (CAD). However, up to 30% EST results are either inconclusive or non-diagnostic, which results in significant resource wastage. Our aim was to build machine learning (ML) based models, using patients demographic (age, sex) and pre-test clinical information (reason for performing test, medications, blood pressure, heart rate, and resting electrocardiogram), capable of predicting EST results beforehand including those with inconclusive or non-diagnostic results.

METHODS

A total of 30,710 patients (mean age 54.0 years, 69% male) were included in the study with 25% randomly sampled in the test set, and the remaining samples were split into a train and validation set with a ratio of 9:1. We constructed different ML models from pre-test variables and compared their discriminant power using the area under the receiver operating characteristic curve (AUC).

RESULTS

A network of Oblivious Decision Trees provided the best discriminant power (AUC=0.83, sensitivity=69%, specificity=0.78%) for predicting inconclusive EST results. A total of 2010 inconclusive ESTs were correctly identified in the testing set.

CONCLUSIONS

Our ML model, developed using demographic and pre-test clinical information, can accurately predict EST results and could be used to identify patients with inconclusive or non-diagnostic results beforehand. Our system could thus be used as a personalised decision support tool by clinicians for optimizing the diagnostic test selection strategy for CAD patients and to reduce healthcare expenditure by reducing nondiagnostic or inconclusive ESTs.

Sixty years of the Bruce protocol: reappraising the contemporary role of exercise stress testing with electrocardiographic monitoring

The cardiovascular response to exercise has long been a focus of interest. Over a century ago, the first descriptions of electrocardiographic changes occurring during exercise highlighted the possible relevance of this dynamic assessment. In this background, the inception of the Bruce protocol circa 60 years ago allowed for a major leap in this field by providing a standardized framework with which to address this issue, by means of an integrated and structured methodology. Since then, exercise stress testing with electrocardiographic monitoring (ExECG) has become one of the most widely appraised tests in cardiovascular medicine. Notably, past few decades have been profoundly marked by substantial advances in the approach to cardiovascular disease, challenging prior notions concerning both its physiopathology and overall management. Among these, the ever-evolving presentations of cardiovascular disease coupled with the development and implementation of several novel diagnostic modalities (both invasive and noninvasive) has led to a shifting paradigm in the application of ExECG. This technique, however, has continuously shown to be of added value across various momentums of the cardiovascular continuum, as depicted in several contemporary guidelines. This review provides a pragmatical reflexion on the development of ExECG, presenting a comprehensive overview concerning the current role of this modality, its challenges, and its future perspectives.

Precision epigenetics provides a scalable pathway for improving coronary heart disease care globally

Coronary heart disease (CHD) is the world's leading cause of death. Up to 90% of all CHD deaths are preventable, but effective prevention of this mortality requires more scalable, precise methods for assessing CHD status and monitoring treatment response. Unfortunately, current diagnostic methods have barriers to implementation, particularly in rural areas and lower-income countries. This gap may be bridged by highly scalable advances in DNA methylation testing methods and artificial intelligence. Herein, we review prior studies of CHD related to methylation alone and in combination with other biovariables. We compare these new methods with established methods for diagnosing CHD. Finally, we outline pathways through which methylation-based testing methods may allow the democratization of improved methods for assessing CHD globally.

Natural language processing for identification of hypertrophic cardiomyopathy patients from cardiac magnetic resonance reports

Background

Cardiac magnetic resonance (CMR) imaging is important for diagnosis and risk stratification of hypertrophic cardiomyopathy (HCM) patients. However, collection of information from large numbers of CMR reports by manual review is time-consuming, error-prone and costly. Natural language processing (NLP) is an artificial intelligence method for automated extraction of information from narrative text including text in CMR reports in electronic health records (EHR). Our objective was to assess whether NLP can accurately extract diagnosis of HCM from CMR reports.

Methods

An NLP system with two tiers was developed for information extraction from narrative text in CMR reports; the first tier extracted information regarding HCM diagnosis while the second extracted categorical and numeric concepts for HCM classification. We randomly allocated 200 HCM patients with CMR reports from 2004 to 2018 into training (100 patients with 185 CMR reports) and testing sets (100 patients with 206 reports).

Results

NLP algorithms demonstrated very high performance compared to manual annotation. The algorithm to extract HCM diagnosis had accuracy of 0.99. The accuracy for categorical concepts included HCM morphologic subtype 0.99, systolic anterior motion of the mitral valve 0.96, mitral regurgitation 0.93, left ventricular (LV) obstruction 0.94, location of obstruction 0.92, apical pouch 0.98, LV delayed enhancement 0.93, left atrial enlargement 0.99 and right atrial enlargement 0.98. Accuracy for numeric concepts included maximal LV wall thickness 0.96, LV mass 0.99, LV mass index 0.98, LV ejection fraction 0.98 and right ventricular ejection fraction 0.99.

Conclusions

NLP identified and classified HCM from CMR narrative text reports with very high performance.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-022-02017-y.

Automated interpretation of stress echocardiography reports using natural language processing

Aims

Stress echocardiography (SE) findings and interpretations are commonly documented in free-text reports. Reusing SE results requires laborious manual reviews. This study aimed to develop and validate an automated method for abstracting SE reports in a large cohort.

Methods and results

This study included adult patients who had SE within 30 days of their emergency department visit for suspected acute coronary syndrome in a large integrated healthcare system. An automated natural language processing (NLP) algorithm was developed to abstract SE reports and classify overall SE results into normal, non-diagnostic, infarction, and ischaemia categories. Randomly selected reports (n = 140) were double-blindly reviewed by cardiologists to perform criterion validity of the NLP algorithm. Construct validity was tested on the entire cohort using abstracted SE data and additional clinical variables. The NLP algorithm abstracted 6346 consecutive SE reports. Cardiologists had good agreements on the overall SE results on the 140 reports: Kappa (0.83) and intraclass correlation coefficient (0.89). The NLP algorithm achieved 98.6% specificity and negative predictive value, 95.7% sensitivity, positive predictive value, and F-score on the overall SE results and near-perfect scores on ischaemia findings. The 30-day acute myocardial infarction or death outcomes were highest among patients with ischaemia (5.0%), followed by infarction (1.4%), non-diagnostic (0.8%), and normal (0.3%) results. We found substantial variations in the format and quality of SE reports, even within the same institution.

Conclusions

Natural language processing is an accurate and efficient method for abstracting unstructured SE reports. This approach creates new opportunities for research, public health measures, and care improvement.

Automated abstraction of myocardial perfusion imaging reports using natural language processing

BACKGROUND

Findings and interpretations of myocardial perfusion imaging (MPI) studies are documented in free-text MPI reports. MPI results are essential for research, but manual review is prohibitively time consuming. This study aimed to develop and validate an automated method to abstract MPI reports.

METHODS

We developed a natural language processing (NLP) algorithm to abstract MPI reports. Randomly selected reports were double-blindly reviewed by two cardiologists to validate the NLP algorithm. Secondary analyses were performed to describe patient outcomes based on abstracted-MPI results on 16,957 MPI tests from adult patients evaluated for suspected ACS.

RESULTS

The NLP algorithm achieved high sensitivity (96.7%) and specificity (98.9%) on the MPI categorical results and had a similar degree of agreement compared to the physician reviewers. Patients with abnormal MPI results had higher rates of 30-day acute myocardial infarction or death compared to patients with normal results. We identified issues related to the quality of the reports that not only affect communication with referring physicians but also challenges for automated abstraction.

CONCLUSION

NLP is an accurate and efficient strategy to abstract results from the free-text MPI reports. Our findings will facilitate future research to understand the benefits of MPI studies but requires validation in other settings.

Systematic review of current natural language processing methods and applications in cardiology

Natural language processing (NLP) is a set of automated methods to organise and evaluate the information contained in unstructured clinical notes, which are a rich source of real-world data from clinical care that may be used to improve outcomes and understanding of disease in cardiology. The purpose of this systematic review is to provide an understanding of NLP, review how it has been used to date within cardiology and illustrate the opportunities that this approach provides for both research and clinical care. We systematically searched six scholarly databases (ACM Digital Library, Arxiv, Embase, IEEE Explore, PubMed and Scopus) for studies published in 2015-2020 describing the development or application of NLP methods for clinical text focused on cardiac disease. Studies not published in English, lacking a description of NLP methods, non-cardiac focused and duplicates were excluded. Two independent reviewers extracted general study information, clinical details and NLP details and appraised quality using a checklist of quality indicators for NLP studies. We identified 37 studies developing and applying NLP in heart failure, imaging, coronary artery disease, electrophysiology, general cardiology and valvular heart disease. Most studies used NLP to identify patients with a specific diagnosis and extract disease severity using rule-based NLP methods. Some used NLP algorithms to predict clinical outcomes. A major limitation is the inability to aggregate findings across studies due to vastly different NLP methods, evaluation and reporting. This review reveals numerous opportunities for future NLP work in cardiology with more diverse patient samples, cardiac diseases, datasets, methods and applications.

Identifying Cases of Shoulder Injury Related to Vaccine Administration (SIRVA) in the United States: Development and Validation of a Natural Language Processing Method (Preprint)

Background

Shoulder injury related to vaccine administration (SIRVA) accounts for more than half of all claims received by the National Vaccine Injury Compensation Program. However, due to the difficulty of finding SIRVA cases in large health care databases, population-based studies are scarce.

Objective

The goal of the research was to develop a natural language processing (NLP) method to identify SIRVA cases from clinical notes.

Methods

We conducted the study among members of a large integrated health care organization who were vaccinated between April 1, 2016, and December 31, 2017, and had subsequent diagnosis codes indicative of shoulder injury. Based on a training data set with a chart review reference standard of 164 cases, we developed an NLP algorithm to extract shoulder disorder information, including prior vaccination, anatomic location, temporality and causality. The algorithm identified 3 groups of positive SIRVA cases (definite, probable, and possible) based on the strength of evidence. We compared NLP results to a chart review reference standard of 100 vaccinated cases. We then applied the final automated NLP algorithm to a broader cohort of vaccinated persons with a shoulder injury diagnosis code and performed manual chart confirmation on a random sample of NLP-identified definite cases and all NLP-identified probable and possible cases.

Results

In the validation sample, the NLP algorithm had 100% accuracy for identifying 4 SIRVA cases and 96 cases without SIRVA. In the broader cohort of 53,585 vaccinations, the NLP algorithm identified 291 definite, 124 probable, and 52 possible SIRVA cases. The chart-confirmation rates for these groups were 95.5% (278/291), 67.7% (84/124), and 17.3% (9/52), respectively.

Conclusions

The algorithm performed with high sensitivity and reasonable specificity in identifying positive SIRVA cases. The NLP algorithm can potentially be used in future population-based studies to identify this rare adverse event, avoiding labor-intensive chart review validation.

Automated Identification and Extraction of Exercise Treadmill Test Results

Background

Noninvasive cardiac tests, including exercise treadmill tests (ETTs), are commonly utilized in the evaluation of patients in the emergency department with suspected acute coronary syndrome. However, there are ongoing debates on their clinical utility and cost‐effectiveness. It is important to be able to use ETT results for research, but manual review is prohibitively time‐consuming for large studies. We developed and validated an automated method to interpret ETT results from electronic health records. To demonstrate the algorithm's utility, we tested the associations between ETT results with 30‐day patient outcomes in a large population.

Methods and Results

A retrospective analysis of adult emergency department encounters resulting in an ETT within 30 days was performed. A set of randomly selected reports were double‐blind reviewed by 2 physicians to validate a natural language processing algorithm designed to categorize ETT results into normal, ischemic, nondiagnostic, and equivocal categories. Natural language processing then searched and categorized results of 5214 ETT reports. The natural language processing algorithm achieved 96.4% sensitivity and 94.8% specificity in identifying normal versus all other categories. The rates of 30‐day death or acute myocardial infarction varied (P<0.001) by categories for normal (0.08%), ischemic (1.9%), nondiagnostic (0.77%), and equivocal (0.58%) groups achieving good discrimination (C‐statistic, 0.81; 95% CI, 0.7–0.92).

Conclusions

Natural language processing is an accurate and efficient strategy to facilitate large‐scale outcome studies of noninvasive cardiac tests. We found that most patients are at low risk and have normal ETT results, while those with abnormal, nondiagnostic, or equivocal results have slightly higher risks and warrant future investigation.