The interobserver agreement of the HEART-score, a multicentre prospective study

Influence of Patient and Clinician Gender on Emergency Department HEART Scores: A Secondary Analysis of a Prospective Observational Trial

STUDY OBJECTIVE

Clinical decision aids can decrease health care disparities. However, many clinical decision aids contain subjective variables that may introduce clinician bias. The HEART score is a clinical decision aid that estimates emergency department (ED) patients' cardiac risk. We sought to explore patient and clinician gender's influence on HEART scores.

METHODS

In this secondary analysis of a prospective observational trial, we examined a convenience sample of adult ED patients at one institution presenting with acute coronary syndrome symptoms. We compared ED clinician-generated HEART scores with researcher-generated HEART scores blinded to patient gender. The primary outcome was agreement between clinician and researcher HEART scores by patient gender overall and stratified by clinician gender. Analyses used difference-in-difference (DiD) for continuous score and prevalence-adjusted, bias-adjusted Kappa (PABAK) for binary (low versus moderate/high risk) score comparison.

RESULTS

All 336 clinician-patient pairs from the original study were included. In total, 47% (158/336) of patients were women, and 52% (174/336) were treated by a woman clinician. The DiD between clinician and researcher HEART scores among men versus women patients was 0.24 (95% CI -0.01 to 0.48). Compared with researchers, men clinicians assigned a higher score to men versus women patients (DiD 0.51 [95% CI 0.16 to 0.87]), whereas women clinicians did not (DiD 0.00 [95% CI -0.33 to 0.33]). Agreement was the highest among women clinicians (PABAK 0.72; 95% CI 0.61 to 0.81) and lowest among men clinicians assessing men patients (PABAK 0.47; 95% CI 0.29 to 0.66).

CONCLUSION

Patient and clinician gender may influence HEART scores. Researchers should strive to understand these influences in developing and implementing this and other clinical decision aids.

Assessment of the Diagnostic Accuracy and Reliability of the HEART Score Calculated by Ambulance Nurses Versus Emergency Physicians

Chest pain is a common complaint for consultation of emergency medical services worldwide. Currently, ambulance nurses (AN) base their decision to transport a patient to the hospital on their own professional experience. The HEART score could improve prehospital risk stratification and patient treatment. The aim of this study is to investigate the interrater reliability and predictive accuracy of the HEART score between AN and emergency physicians (EP). A retrospective analysis on data of 569 patients 18 years and older included in two prehospital HEART score studies. The endpoints are interrater reliability (intraclass correlation [ICC]) and predictive accuracy for major adverse cardiac events within 30 days of the HEART score calculated by AN versus EP. Predictive accuracy is sensitivity, specificity, positive predicted value (PPV) and negative predicted value (NPV). Interrater reliability was good for total HEART score (ICC 0.78; 95% CI 0.75-0.81). However, focusing on the decision to transport a patient, the ICC dropped to 0.62 (95% CI 0.62-0.70). History and Risk factors caused the most variability. Predictive accuracy of HEART differed between AN and EP. The HEART score calculated by AN was sensitivity 91%, specificity 38%, PPV 26%, and NPV 95%. The HEART score calculated by EP was sensitivity 98%, specificity 32%, PPV -26%, and NPV 99%. With a cut-off value of 0-2 for a low HEART score, predictive accuracy significantly improved for the HEART score calculated by AN: sensitivity 98%, specificity 18%, PPV 22%, and NPV 98%. Our study shows a moderate interrater reliability and lower predictive accuracy of a HEART score calculated by AN versus EP. AN underestimate the risk of patients with acute chest pain, with the largest discrepancies in the elements History and Risk factors. Reconsidering the cut-off values of the low-risk HEART category, as well as a carefully developed training program, will possibly lead to a higher interrater reliability of the HEART score and higher predictive accuracy used by AN.

Prehospital stratification and prioritisation of non-ST-segment elevation acute coronary syndrome patients (NSTEACS): the MARIACHI scale

OBJECTIVE

The objective of this study was to develop and validate a risk scale (MARIACHI) for patients classified as non-ST-segment elevation acute coronary syndrome (NSTEACS) in a prehospital setting with the ability to identify patients at an increased risk of mortality at an early stage.

METHODS

A retrospective observational study conducted in Catalonia over two periods: 2015-2017 (development and internal validation cohort) and Aug 2018-Jan 2019 (external validation cohort). We included patients classified as prehospital NSTEACS, assisted by an advanced life support unit and requiring hospital admission. The primary outcome was in-hospital mortality. Cohorts were compared using logistic regression and a predictive model was created using bootstrapping techniques.

RESULTS

The development and internal validation cohort included 519 patients. The model is composed of five variables associated with hospital mortality: age, systolic blood pressure, heart rate > 95 bpm, Killip-Kimball III-IV and ST depression ≥ 0.5 mm. The model showed good overall performance (Brier = 0.043) and consistency in discrimination (AUC 0.88, 95% CI 0.83-0.92) and calibration (slope = 0.91; 95% CI 0.89-0.93). We included 1316 patients for the external validation sample. There was no difference in discrimination (AUC 0.83, 95% CI 0.78-0.87; DeLong Test p = 0.071), but there was in calibration (p < 0.001), so it was recalibrated. The finally model obtained was stratified and scored into three groups according to the predicted risk of patient in-hospital mortality: low risk: < 1% (-8 to 0 points), moderate risk: 1-5% (+ 1 to + 5 points) and high risk: > 5% (6-12 points).

CONCLUSION

The MARIACHI scale showed correct discrimination and calibration to predict high-risk NSTEACS. Identification of high-risk patients may help with treatment and low referral decisions at the prehospital level.

Performances of HEART score to predict 6-month prognostic of emergency department patients with chest pain: a retrospective cohort analysis

Chest pain is a frequent cause of patient admissions in emergency departments (EDs). Clinical scores can help in the management of chest pain patients with an undefined impact on the appropriateness of hospitalization or discharge when compared to usual care. The aim of this study was to assess the performances of the HEART score to predict the 6-month prognostic of patients presenting to the ED of a tertiary referral university hospital with non-traumatic chest pain. From 7040 patients presenting with chest pain from 1 January 2015 to 31 December 2017, after applying exclusion criteria (ST-segment elevation >1 mm, shock, absence of telephone number) we selected a sample of 20% chosen randomly. We retrospectively assessed the clinical course, definitive diagnosis, and HEART score according to ED final report. Follow-up was made by telephone interview with discharged patients. In hospitalized patients, clinical records were analyzed to evaluate major adverse cardiac events (MACE) incidence. The primary endpoint was MACE, comprising cardiovascular death, myocardial infarction, or unscheduled revascularization at 6 months. We assessed the diagnostic performance of the HEART score in ruling out MACE at 6 months. We also assessed the performance of ED usual care in the management of chest pain patients. Of 1119 screened, 1099 were included for analysis after excluding patients lost to follow-up; 788 patients (71.70%) had been discharged and 311 (28.30%) were hospitalized. Incident MACE was 18.3% (n = 205). The HEART score was retrospectively calculated in 1047 patients showing increasing MACE incidence according to risk category (0.98% for low risk, 38.02% for intermediate risk, and 62.21% for high risk). Low-risk category allowed to safely exclude MACE at 6 months with a negative predictive value (NPV) of 99%. Usual care diagnostic performance showed 97.38% sensitivity, 98.24% specificity, 95.5% positive predictive value, and 99% NPV, with an overall accuracy of 98.00%. In ED patients with chest pain, a low HEART score is associated with a very low risk of MACE at 6 months.

A Closer Look at the HEART Score

The history, electrocardiogram, age, risk factors, and troponin (HEART) score is currently a widely used tool for acute chest pain risk stratification. Relatively soon after its inception in 2008, a number of validation studies on the HEART score showed it to be superior to Thrombolysis in Myocardial Infarction (TIMI) and Global Registry of Acute Coronary Events (GRACE) scores and at least as accurate to other existing scores for predicting short-term major adverse cardiovascular events (MACEs). However, partly due to its focus on simplicity, the HEART score has some limitations. In this article we review how the HEART score has evolved and taken on various modifications to circumvent some of its limitations. We also highlight the strength of the HEART score in comparison with other risk stratification tools and the current guidelines.

SVEAT score outperforms HEART score in patients admitted to a chest pain observation unit

BACKGROUND

Timely and accurate identification of subgroup at risk for major adverse cardiovascular events among patients presenting with acute chest pain remains a challenge. Currently available risk stratification scores are suboptimal. Recently, a new scoring system called the Symptoms, history of Vascular disease, Electrocardiography, Age, and Troponin (SVEAT) score has been shown to outperform the History, Electrocardiography, Age, Risk factors and Troponin (HEART) score, one of the most used risk scores in the United States.

AIM

To assess the potential usefulness of the SVEAT score as a risk stratification tool by comparing its performance to HEART score in chest pain patients with low suspicion for acute coronary syndrome and admitted for overnight observation.

METHODS

We retrospectively reviewed medical records of 330 consecutive patients admitted to our clinical decision unit for acute chest pain between January 1^st to April 17^th, 2019. To avoid potential biases, investigators assigned to calculate the SVEAT, and HEART scores were blinded to the results of 30-d combined endpoint of death, acute myocardial infarction or confirmed coronary artery disease requiring revascularization or medical therapy [30-d major adverse cardiovascular event (MACE)]. An area under receiving-operator characteristic curve (AUC) for each score was then calculated. C-statistic and logistic model were used to compare predictive performance of the two scores.

RESULTS

A 30-d MACE was observed in 11 patients (3.33% of the subjects). The AUC of SVEAT score (0.8876, 95%CI: 0.82-0.96) was significantly higher than the AUC of HEART score (0.7962, 95%CI: 0.71-0.88), P = 0.03. Using logistic model, SVEAT score with cut-off of 4 or less significantly predicts 30-d MACE (odd ratio 1.52, 95%CI: 1.19-1.95, P = 0.001) but not the HEART score (odd ratio 1.29, 95%CI: 0.78-2.14, P = 0.32).

CONCLUSION

The SVEAT score is superior to the HEART score as a risk stratification tool for acute chest pain in low to intermediate risk patients.

Interrater agreement of the HEART score history component: A chart review study

Study objectives

This study investigated the interrater reliability of the history component of the HEART (history, electrocardiogram, age, risk, troponin) score between physicians in emergency medicine (EM) and internal medicine (IM) at 1 tertiary‐care center.

Methods

We conducted a retrospective, secondary analysis of 60 encounters selected randomly from a database of 417 patients with chest pain presenting from January to June 2016 to an urban tertiary‐care center. A total of 4 raters (1 EM attending, 1 EM resident, 1 IM attending, and 1 IM resident) scored the previously abstracted history data from these encounters.

The primary outcome was the interrater agreement of HEART score history components, as measured by kappa coefficient, between EM and IM attending physicians. Secondary outcomes included the agreement between attending and resident physicians, overall agreement, pairwise percent agreement, and differences in scores assigned.

Results

The kappa value for the EM attending physician and IM attending physician was 0.33 with 55% agreement. Interrater agreement of the other pairs was substantial between EM attending and resident but was otherwise fair to moderate. Percent agreement between the other pairs ranged from 48.3% to 80%. There was a significant difference in scores assigned and the subgroup in which there was disagreement between the raters demonstrated significantly higher scores by the EM attending and resident when compared to the IM attending.

Conclusion

This study demonstrates fair agreement between EM and IM attending physicians in the history component of the HEART score with significantly higher scores by the EM attending physician in cases of disagreement at 1 tertiary‐care center.

Role of HEART score in prediction of coronary artery disease and major adverse cardiac events in patients presenting with chest pain

Introduction. Chest pain (CP) diagnostics accuracy remains debatable for both general practitioners (GP) or emergency department (ED) physicians for patients in HEART score (HS) low- and intermediate-risk groups which prompted us to review our electronic database for all patients admitted via our center?s ED during 2014 to 2020 for CP and suspect acute coronary syndrome. Methods. Patients were divided in function of low- or intermediate-risk HS and assessed during a three month follow up for angiogram results, MACE, lab results and echo parameters. Results. Of 585 patients included, low-risk HS group (21,4%, 36% were women) had significant coronary disease on angiogram in 68%, while for intermediate-risk HS group (78.6%, with 32.6% women) it was for 18.4% of patients (p < 0,0005). Area under the ROC curve of HS in detecting patients with ischemic heart disease as a cause of CP was 0.771 (95% CI:0.772-0.820) with best cut-off point HS was calculated at 3.5. Sensitivity and specificity were 89.2% and 57.6% respectively. Adjusting for sex, lab results and HS, AUROC curve of this model was 0.828 (95% CI:0.786-0.869; p < 0,0005) with cut-off of 77.95. Sensitivity and specificity were 84,9% and 68% respectively. In the three-month follow-up post-discharge, there was a significant difference in MACE between groups (low-vs. intermediate-risk HS was 3.4 vs. 16.7% p < 0.05). Conclusion. HS for our CP patients admitted via our ED by GP and ED physicians? referral, provides a quick and reliable prediction of ischemic heart disease and MACE.

A Prospective Evaluation of Clinical HEART Score Agreement, Accuracy, and Adherence in Emergency Department Chest Pain Patients

STUDY OBJECTIVE

The HEART score is a risk stratification aid that may safely reduce chest pain admissions for emergency department patients. However, differences in interpretation of subjective components potentially alters the performance of the score. We compared agreement between HEART scores determined during clinical practice with research-generated scores and estimated their accuracy in predicting 30-day major adverse cardiac events.

METHODS

We prospectively enrolled adult ED patients with symptoms concerning for acute coronary syndrome at a single tertiary center. ED clinicians submitted their clinical HEART scores during the patient encounter. Researchers then independently interviewed patients to generate a research HEART score. Patients were followed by phone and chart review for major adverse cardiac events. Weighted kappa; unweighted Cohen's kappa; prevalence-adjusted, bias-adjusted kappa (PABAK); and test probabilities were calculated.

RESULTS

From November 2016 to June 2019, 336 patients were enrolled, 261 (77.7%) were admitted, and 30 (8.9%) had major adverse cardiac events. Dichotomized HEART score agreement was 78% (kappa 0.48, 95% confidence interval [CI] 0.37 to 0.58; PABAK 0.57, 95% CI 0.48 to 0.65) with the lowest agreement in the history (72%; WK 0.14, 95% CI 0.06 to 0.22) and electrocardiogram (85%; WK 0.4, 95% CI 0.3 to 0.49) components. Compared with researchers, clinicians had 100% sensitivity (95% CI 88.4% to 100%) (versus 86.7%, 95% CI 69.3% to 96.2%) and 27.8% specificity (95% CI 22.8% to 33.2%) (versus 34.6%, 95% CI 29.3% to 40.3%) for major adverse cardiac events. Four participants with low research HEART scores had major adverse cardiac events.

CONCLUSION

ED clinicians had only moderate agreement with research HEART scores. Combined with uncertainties regarding accuracy in predicting major adverse cardiac events, we urge caution in the widespread use of the HEART score as the sole determinant of ED disposition.