Optimisation of telephone triage of callers with symptoms suggestive of acute cardiovascular disease in out-of-hours primary care: observational design of the Safety First study

Shortness of breath as a diagnostic factor for acute coronary syndrome in male and female callers to out-of-hours primary care

OBJECTIVE

Chest discomfort and shortness of breath (SOB) are key symptoms in patients with acute coronary syndrome (ACS). It is, however, unknown whether SOB is valuable for recognising ACS during telephone triage in the out-of-hours primary care (OHS-PC) setting.

METHODS

A cross-sectional study performed in the Netherlands. Telephone triage conversations were analysed of callers with chest discomfort who contacted the OHS-PC between 2014 and 2017, comparing patients with SOB with those who did not report SOB. We determine the relation between SOB and (1) High urgency allocation, (2) ACS and (3) ACS or other life-threatening diseases.

RESULTS

Of the 2195 callers with chest discomfort, 1096 (49.9%) reported SOB (43.7% men, 56.3% women). In total, 15.3% men (13.2% in those with SOB) and 8.4% women (9.2% in those with SOB) appeared to have ACS. SOB compared with no SOB was associated with high urgency allocation (75.9% vs 60.8%, OR: 2.03; 95% CI 1.69 to 2.44, multivariable OR (mOR): 2.03; 95% CI 1.69 to 2.44), but not with ACS (10.9% vs 12.0%; OR: 0.90; 95% CI 0.69 to 1.17, mOR: 0.91; 95% CI 0.70 to 1.19) or 'ACS or other life-threatening diseases' (15.0% vs 14.1%; OR: 1.07; 95% CI 0.85 to 1.36, mOR: 1.09; 95% CI 0.86 to 1.38). For women the relation with ACS was 9.2% vs 7.5%, OR: 1.25; 95% CI 0.83 to 1.88, and for men 13.2% vs 17.4%, OR: 0.72; 95% CI 0.51 to 1.02. For 'ACS or other life-threatening diseases', this was 13.0% vs 8.5%, OR: 1.60; 95% CI 1.10 to 2.32 for women, and 7.5% vs 20.8%, OR: 0.81; 95% CI 0.59 to 1.12 for men.

CONCLUSIONS

Men and women with chest discomfort and SOB who contact the OHS-PC more often receive high urgency than those without SOB. This seems to be adequate in women, but not in men when considering the risk of ACS or other life-threatening diseases.

Development and validation of a prediction rule for patients suspected of acute coronary syndrome in primary care: a cross-sectional study

OBJECTIVE

To develop and validate a symptom-based prediction rule for early recognition of acute coronary syndrome (ACS) in patients with acute chest discomfort who call out-of-hours services for primary care (OHS-PC).

DESIGN

Cross-sectional study. A diagnostic prediction rule was developed with multivariable regression analyses. All models were validated with internal-external cross validation within seven OHS-PC locations. Both age and sex were analysed as statistical interaction terms, applying for age non-linear effects.

SETTING

Seven OHS-PC in the Netherlands.

PARTICIPANTS

2192 patients who called OHS-PC for acute chest discomfort (pain, pressure, tightness or discomfort) between 2014 and 2017. Backed up recordings of telephone triage conversations were analysed.

PRIMARY AND SECONDARY OUTCOMES MEASURES

Diagnosis of ACS retrieved from the patient's medical records in general practice, including hospital specialists discharge letters. Performance of the prediction rules was calculated with the c-statistic and the final model was chosen based on net benefit analyses.

RESULTS

Among the 2192 patients who called the OHS-PC with acute chest discomfort, 8.3% females and 15.3% males had an ACS. The final diagnostic model included seven predictors (sex, age, acute onset of chest pain lasting less than 12 hours, a pressing/heavy character of the pain, radiation of the pain, sweating and calling at night). It had an adjusted c-statistic of 0.77 (95% CI 0.74 to 0.79) with good calibration.

CONCLUSION

The final prediction model for ACS has good discrimination and calibration and shows promise for replacing the existing telephone triage rules for patients with acute chest discomfort in general practice and OHS-PC.

TRIAL REGISTRATION NUMBER

NTR7331.

Missed Acute Coronary Syndrome During Telephone Triage at Out-of-Hours Primary Care: Lessons From A Case-Control Study

OBJECTIVES

Serious adverse events at out-of-hours services in primary care (OHS-PC) are rare, and the most often concern is missed acute coronary syndrome (ACS). Previous studies on serious adverse events mainly concern root cause analyses, which highlighted errors in the telephone triage process but are hampered by hindsight bias. This study compared the recorded triage calls of patients with chest discomfort contacting the OHS-PC in whom an ACS was missed (cases), with triage calls involving matched controls with chest discomfort but without a missed ACS (controls), with the aim to assess the predictors of missed ACS.

METHODS

A case-control study with data from 2013 to 2017 of 9 OHS-PC in the Netherlands. The cases were matched 1:8 with controls based on age and sex. Clinical, patient, and call characteristics were univariably assessed, and general practitioner experts evaluated the triage while blinded to the final diagnosis or the case-control status.

RESULTS

Fifteen missed ACS calls and 120 matched control calls were included. Cases used less cardiovascular medication (38.5% versus 64.1%, P = 0.05) and more often experienced pain other than retrosternal chest pain (63.3% versus 24.7%, P = 0.02) compared with controls. Consultation of the supervising general practitioner (86.7% versus 49.2%, P = 0.02) occurred more often in cases than in controls. Experts rated the triage of cases more often as "poor" (33.3% versus 10.9%, P = 0.001) and "unsafe" (73.3% versus 22.5%, P < 0.001) compared with controls.

CONCLUSIONS

To facilitate learning from serious adverse events in the future, these should also be bundled and carefully assessed without hindsight bias and within the context of "normal" clinical practice.

Gender-stratified analyses of symptoms associated with acute coronary syndrome in telephone triage: a cross-sectional study

OBJECTIVES

To identify clinical variables that are associated with the diagnosis acute coronary syndrome (ACS) in women and men with chest discomfort who contact out-of-hours primary care (OHS-PC) by telephone, and to explore whether there are indications whether these variables differ among women and men.

DESIGN

Cross-sectional study in which we compared patient and call characteristics of triage call recordings between women with and without ACS, and men with and without ACS.

SETTING

Nine OHS-PC in the Netherlands.

PARTICIPANTS

993 women and 802 men who called OHS-PC for acute chest discomfort (pain, pressure, tightness or discomfort) between 2014 and 2016.

PRIMARY OUTCOME MEASURE

Diagnosis of ACS retrieved from the patient's medical record in general practice, including hospital specialists' discharge letters.

RESULTS

Among 1795 patients (mean age 58.8 (SD 19.5) years, 55.3% women), 15.0% of men and 8.6% of women had an ACS. In both sexes, retrosternal chest pain was associated with ACS (women with ACS vs without 62.3% vs 40.3%, p=0.002; men with ACS vs without 52.5% vs 39.7%, p=0.032; gender interaction, p=0.323), as was pressing/heavy/tightening pain (women 78.6% vs 61.5%, p=0.011; men 82.1% vs 57.4%, p=<0.001; gender interaction, p=0.368) and radiation to the arm (women 75.6% vs 45.9%, p<0.001; men 56.0% vs 34.8%, p<0.001; gender interaction, p=0.339). Results indicate that only in women were severe pain (65.4% vs 38.1%, p=0.006; gender interaction p=0.007) and radiation to jaw (50.0% vs 22.9%, p=0.007; gender interaction p=0.015) associated with ACS.Ambulances were dispatched equally in women (72.9%) and men with ACS (70.0%).

CONCLUSION

Our results indicate there were more similarities than differences in symptoms associated with the diagnosis ACS for women and men. Important exceptions were pain severity and radiation of pain in women. Whether these differences have an impact on predicting ACS needs to be further investigated with multivariable analyses.

TRIAL REGISTRATION NUMBER

NTR7331.

Call Characteristics of Patients Suspected of Transient Ischemic Attack (TIA) or Stroke During Out-of-Hours Service: A Comparison Between Men and Women

Background: In the Netherlands, a digital decision support system for telephone triage at out-of-hours services in primary care (OHS-PC) is used. Differences in help-seeking behavior between men and women when transient ischemic attack (TIA) or stroke is suspected could potentially affect telephone triage and allocation of urgency.

Aim: To assess patient and call characteristics and allocated urgencies between women and men who contacted OHS-PC with suspected TIA/stroke.

Methods: A cross-sectional study of 1,266 telephone triage recordings of subjects with suspected neurological symptoms calling the OHS-PC between 2014 and 2016. The allocated urgencies were derived from the electronic medical records of the OHS-PC and the final diagnosis from the patient's own general practitioner, including diagnoses based on hospital specialist letters.

Results: Five hundred forty-six men (mean age = 67.3 ± 17.1) and 720 women (mean age = 69.6 ± 19.5) were included. TIA/stroke was diagnosed in 294 men (54%) (mean age = 72.3 ± 13.6) and 366 women (51%) (mean age = 78.0 ± 13.8). In both genders, FAST (face-arm-speech test) symptoms were common in TIA/stroke (men 78%, women 82%) but also in no TIA/stroke (men 63%, women 62%). Men with TIA/stroke had shorter call durations than men without TIA/stroke (7.10 vs. 8.20 min, p = 0.001), whereas in women this difference was smaller and not significant (7.41 vs. 7.56 min, p = 0.41). Both genders were allocated high urgency in 75% of the final TIA/stroke cases.

Conclusion: Overall, patient and call characteristics are mostly comparable between men and women, and these only modestly assist in identifying TIA/stroke. There were no gender differences in allocated urgencies after telephone triage in patients with TIA/stroke.

Income-related differences in out-of-hours primary care telephone triage using national registration data

BACKGROUND

Telephone triage is used to facilitate efficient and adequate acute care allocation, for instance in out-of-hours primary care services (OPCSs). Remote assessment of health problems is challenging and could be impeded by a patient's ambiguous formulation of his or her healthcare need. Socioeconomically vulnerable patients may experience more difficulty in expressing their healthcare need. We aimed to assess whether income differences exist in the patient's presented symptoms, assessed urgency and allocation of follow-up care in OPCS.

METHOD

Data were derived from Nivel Primary Care Database encompassing electronic health record data of 1.3 million patients from 28 OPCSs in 2017 in the Netherlands. These were linked to sociodemographic population registry data. Multilevel logistic regression analyses (contacts clustered in patients), adjusted for patient characteristics (eg, age, sex), were conducted to study associations of symptoms, urgency assessment and follow-up care with patients' income (standardised for household size as socioeconomic status (SES) indicator).

RESULTS

The most frequently presented symptoms deduced during triage slightly differed across SES groups, with a larger relative share of trauma in the high-income groups. No SES differences were observed in urgency assessment. After triage, low income was associated with a higher probability of receiving telephone advice and home visits, and fewer consultations at the OPCS.

CONCLUSIONS

SES differences in the patient's presented symptom and in follow-up in OPCS suggest that the underlying health status and the ability to express care needs affect the telephone triage process . Further research should focus on opportunities to better tailor the telephone triage process to socioeconomically vulnerable patients.

Interactional implications of either/or-questions during telephone triage of callers with chest discomfort in out-of-hours primary care: A conversation analysis

OBJECTIVE

To explore the interactional implications of either/or-questions on the interaction between people who call out-of-hours services in primary care (OHS-PC) and triage nurses who use a decision support tool called the 'Netherlands Triage Standard' (NTS) during telephone triage.

METHODS

A qualitative study of 68 triage conversations at six Dutch OHS-PC. Patients called the OHS-PC with symptoms, e.g. chest discomfort, suggestive of acute coronary syndrome. Using conversation analysis, we identified two categories of multiple-choice either/or-questions that indicated interactional difficulties, shown in hesitation markers within callers' responses.

RESULTS

Our analysis shows that interactional difficulties mainly arise when (i) questions are poorly designed by the triage nurse; or (ii) when the caller's complaints are ambiguously presented reflecting patient's difficulties to verbalize them (e.g. "not feeling well").

CONCLUSION

The way NTS displays key diagnostic options encourages triage nurses to use multiple-choice either/or-questions. More awareness among triage nurses is needed on undesirable implications of either/or-questions on the interaction.

PRACTICE IMPLICATIONS

We recommend changing the NTS display of diagnostic options and to use questions with fewer options in order to decrease the chance of formulating ambiguous questions soliciting unclear responses. Furthermore, asking content questions when complaints are ambiguously formulated may specify the presentation of complaints.

Accuracy of telephone triage in primary care patients with chest discomfort: a cross-sectional study

OBJECTIVE

To assess the accuracy of semi-automatic assisted telephone triage in patients with acute chest discomfort against the diagnosis of acute coronary syndrome (ACS) or other life-threatening events (LTEs).

METHODS

A cross-sectional study was performed of telephone conversations with 2023 patients with acute chest discomfort (pain, pressure, tightness or discomfort) who called out-of-hours services for primary care (OHS-PC) between 2014 and 2016. Sensitivity, specificity, positive and negative predicted values were calculated for a high urgency (patient seen within one hour) against the diagnoses of ACS and other LTEs. Diagnoses were retrieved from the patients' medical records in general practice, including hospital specialists' discharge letters.

RESULTS

Of 2023 patients who called because of chest discomfort, 227 (11.2%) had an ACS (men 14.9%, women 8.2%) and 58 (2.9%) had another LTE (men 3.6%, women 2.3%). The sensitivity and specificity of a high Netherlands Triage System (NTS) urgency allocation against ACS/other LTEs were 0.73 (95% CI 0.68 to 0.78) and 0.43 (95% CI 0.40 to 0.45), respectively. In 13.2% of the calls the triage nurse overruled the NTS urgency, mostly by upscaling (11.0%). The sensitivity and specificity of the final urgency allocation were 0.86 (95% CI 0.81 to 0.90) and 0.34 (95% CI 0.32 to 0.37). The positive and negative predictive values of the final urgency were 0.18 (95% CI 0.17 to 0.19) and 0.94 (95% CI 0.92 to 0.95), respectively.

CONCLUSIONS

The semi-automatic triage NTS tool underestimated the urgency in 27% of patients with ACS/other LTEs. Overruling by triage nurses improved safety, but still 14% of men and women with ACS/other LTEs received too low urgency, while efficiency remained poor.

TRIAL REGISTRATION NUMBER

NTR7331.

Is the time of calling helpful for differentiating transient ischaemic attack and stroke from mimics in primary care out-of-hours services? A cross-sectional study

OBJECTIVES

Telephone triage of patients suspected of transient ischaemic attack (TIA) or stroke is challenging. Both TIA and stroke more likely occur during daytime, with a peak in the morning hours. Thus, the time of calling might be a helpful determinant during telephone triage. We assessed the time of calling in patients with stroke-like symptoms who called the out-of-hours services in primary care (OHS-PC), and evaluated whether the time of calling differed between patients with TIA or stroke compared with those with mimics.

DESIGN

Cross-sectional study.

SETTING

Six OHS-PC locations in the Netherlands.

PARTICIPANTS

1269 telephone triage recordings of patients calling the OHS-PC because of stroke-like symptoms. We collected information on patient characteristics, symptoms, time of calling and urgency allocation. The final diagnosis related to each triage call was based on letters from the neurologist (retrieved from the patient's general practitioner).

PRIMARY AND SECONDARY OUTCOME MEASURES

The primary outcome measures were the time of calling hourly and 4 hourly, and the risk of TIA or stroke/hour. The secondary outcome measure was the risk ratio of TIA or stroke in the morning (08:00-12:00h) versus other hours.

RESULTS

Mean age was 68.6 (SD±18.5) years, 56.9% were women and 50.0% had a TIA or stroke. The risk ratio of TIA or stroke among people calling with stroke-like symptoms between 08:00-12:00h versus other hours was 1.13 (95% CI 1.00 to 1.28, p=0.070). After correction for age and sex, the adjusted risk ratio was 0.94 (95% CI 0.80 to 1.10, p=0.434).

CONCLUSION

In patients who called the OHS-PC because of stroke-like symptoms, the time of calling did not differ between patients with TIA or stroke and patients with mimics.

TRIAL REGISTRATION NUMBER

The Netherlands National Trial Registry (NTR7331).

Accuracy of telephone triage in patients suspected of transient ischaemic attack or stroke: a cross-sectional study

BACKGROUND

The Netherlands Triage Standard (NTS) is a widely used decision support tool for telephone triage at Dutch out-of-hours primary care services (OHS-PC), which, however, has never been validated against clinical outcomes. We aimed to determine the accuracy of the NTS urgency allocation for patients with neurological symptoms suggestive of a transient ischaemic attack (TIA) or stroke, with the clinical outcomes TIA, stroke, and other (neurologic) life-threatening events (LTEs) as the reference.

METHOD

A cross-sectional study of telephone triage recordings of patients with neurological symptoms calling the OHS-PC between 2014 and 2016.The allocated NTS urgencies were derived from the electronic medical records of the OHS-PC. The clinical outcomes were retrieved from the electronic medical records of the patients' own general practitioners. The accuracy of a high NTS urgency allocation (medical help within 3 h) was calculated in terms of sensitivity, specificity, positive and negative predictive values (PPV and NPV) with the clinical outcomes TIA/stroke/other LTEs as the reference.

RESULTS

Of 1269 patients, 635 (50.0%) received the diagnosis TIA/stroke (34.2% TIA/minor stroke, 15.8% major ischaemic or haemorrhagic stroke), and 4.8% other LTEs. For TIA/stroke/other LTEs, the sensitivity and specificity of the NTS urgency allocation were 0.72 (95%CI 0.68-0.75) and 0.48 (95%CI 0.43-0.52), and the PPV and NPV were 0.62 (95%CI 0.60-0.64) and 0.58 (95%CI 0.54-0.62).

CONCLUSIONS

The NTS decision support tool used in Dutch OHS-PC performed poor to moderately regarding safety (sensitivity) and efficiency (specificity) in allocating adequate urgencies to patients with and without TIA/stroke/other LTEs.

TRIAL REGISTRATION

The Netherlands National Trial Register, identification number NTR7331 /Trial NL7134 .

Chest discomfort at night and risk of acute coronary syndrome: cross-sectional study of telephone conversations

BACKGROUND

During telephone triage, it is difficult to assign adequate urgency to patients with chest discomfort. Considering the time of calling could be helpful.

OBJECTIVE

To assess the risk of acute coronary syndrome (ACS) in certain time periods and whether sex influences this risk.

METHODS

Cross-sectional study of 1655 recordings of telephone conversations of patients who called the out-of-hours services primary care (OHS-PC) for chest discomfort. Call time, patient characteristics, symptoms, medical history and urgency allocation of the triage conversations were collected. The final diagnosis of each call was retrieved at the patient's general practice. Absolute numbers of patients with and without ACS were plotted and risks per hour were calculated. The risk ratio of ACS at night (0 to 9 am) was calculated by comparing to the risk at other hours and was adjusted for gender and age.

RESULTS

The mean age of callers was 58.9 (standard deviation ±19.5) years, 55.5% were women and, in total, 199 (12.0%) had an ACS. The crude risk ratio for an ACS at night was 1.80 (confidence interval 1.39-2.34, P < 0.001): 2.33 (1.68-3.22, P < 0.001) for men and 1.29 (0.83-1.99, P = 0.256) for women. The adjusted risk ratio for ACS of all people at night was 1.82 (1.07-3.10, P = 0.039).

CONCLUSIONS

Patients calling the OHS-PC for chest discomfort between 0 and 9 am have almost twice a higher risk of ACS than those calling other hours, a phenomenon more evident in men than in women. At night, dispatching ambulances more 'straightaway' could be considered for these patients with chest discomfort.

TRIAL NUMBER

NTR7331.

Limited reliability of experts' assessment of telephone triage in primary care patients with chest discomfort

OBJECTIVE

Root cause analyses of serious adverse events (SAE) in out-of-hours primary care (OHS-PC) often point to errors in telephone triage. Such analyses are, however, hampered by hindsight bias. We assessed whether experts, blinded to the outcome, recognize (un)safety of triage of patients with chest discomfort, and we quantified inter-rater reliability.

STUDY DESIGN AND SETTING

This is a case-control study with triage recordings from 2013-2017 at OHS-PC. Cases were missed acute coronary syndromes (ACSs, considered as SAE). These cases were age- and gender-matched 1:8 with the controls, sampled from the remainder of people calling for chest discomfort. Fifteen experts listened to the recordings and rated the safety of triage. We calculated sensitivity and specificity of recognizing an ACS and the intraclass correlation.

RESULTS

In total, 135 calls (15 SAE, 120 matched controls) were relistened. The experts identified ACSs with a sensitivity of 0.86 (95% CI: 0.71-0.95) and a specificity of 0.51 (95% CI: 0.43-0.58). Cases were rated significantly more often as unsafe than the controls (73.3% vs. 22.5%, P < 0.001). The inter-rater reliability for safety was poor: ICC 0.16 (95% CI: 0.00-0.32).

CONCLUSIONS

Blinded experts rated calls of missed ACSs more often as unsafe than matched control calls, but with a low level of agreement among the experts.

Rationale and design of a cohort study evaluating triage of acute chest pain in out-of-hours primary care in the Netherlands (TRACE)

The aims of this study are (1) to evaluate the performance of current triage for chest pain; (2) to describe the case mix of patients undergoing triage for chest pain; and (3) to identify opportunities to improve performance of current Dutch triage system for chest pain. Chest pain is a common symptom, and identifying patients with chest pain that require urgent care can be quite challenging. Making the correct assessment is even harder during telephone triage. Temporal trends show that the referral threshold has lowered over time, resulting in overcrowding of first responders and emergency services. While various stakeholders advocate for a more efficient triage system, careful evaluation of the performance of the current triage in primary care is lacking. TRiage of Acute Chest pain Evaluation in primary care (TRACE) is a large cohort study designed to describe the current Dutch triage system for chest pain and subsequently evaluate triage performance in regard to clinical outcomes. The study consists of consecutive patients who contacted the out-of-hours primary care facility with chest pain in the region of Alkmaar, the Netherlands, in 2017, with follow-up for clinical outcomes out to August 2019. The primary outcome of interest is ‘major event’, which is defined as the occurrence of death from any cause, acute coronary syndrome, urgent coronary revascularization, or other high-risk diagnoses in which delay is inadmissible and hospitalization is necessary. We will evaluate the performance of the triage system by assessing the ability of the triage system to correctly classify patients regarding urgency (accuracy), the proportion of safe actions following triage (safety) as well as rightfully deployed ambulances (efficacy). TRACE is designed to describe the current Dutch triage system for chest pain in primary care and to subsequently evaluate triage performance in regard to clinical outcomes.