Developing outcome, process and balancing measures for an emergency department longitudinal patient monitoring system using a modified Delphi

BACKGROUND

Early warning score systems have been widely recommended for use to detect clinical deterioration in patients. The Irish National Emergency Medicine Programme has developed and piloted an emergency department specific early warning score system. The objective of this study was to develop a consensus among frontline healthcare staff, quality and safety staff and health systems researchers regarding evaluation measures for an early warning score system in the Emergency Department.

METHODS

Participatory action research including a modified Delphi consensus building technique with frontline hospital staff, quality and safety staff, health systems researchers, local and national emergency medicine stakeholders was the method employed in this study. In Stage One, a workshop was held with the participatory action research team including frontline hospital staff, quality and safety staff and health systems researchers to gather suggestions regarding the evaluation measures. In Stage Two, an electronic modified-Delphi study was undertaken with a panel consisting of the workshop participants, key local and national emergency medicine stakeholders. Descriptive statistics were used to summarise the characteristics of the panellists who completed the questionnaires in each round. The mean Likert rating, standard deviation and 95% bias-corrected bootstrapped confidence interval for each variable was calculated. Bonferroni corrections were applied to take account of multiple testing. Data were analysed using Stata 14.0 SE.

RESULTS

Using the Institute for Healthcare Improvement framework, 12 process, outcome and balancing metrics for measuring the effectiveness of an ED-specific early warning score system were developed.

CONCLUSION

There are currently no published measures for evaluating the effectiveness of an ED early warning score system. It was possible in this study to develop a suite of evaluation measures using a modified Delphi consensus approach. Using the collective expertise of frontline hospital staff, quality and safety staff and health systems researchers to develop and categorise the initial set of potential measures was an innovative and unique element of this study.

Study protocol for evaluating the implementation and effectiveness of an emergency department longitudinal patient monitoring system using a mixed-methods approach

BACKGROUND

Early detection of patient deterioration is a key element of patient safety as it allows timely clinical intervention and potential rescue, thus reducing the risks of serious patient safety incidents. Longitudinal patient monitoring systems have been widely recommended for use to detect clinical deterioration. However, there is conflicting evidence on whether they improve patient outcomes. This may in part be related to variation in the rigour with which they are implemented and evaluated. This study aims to evaluate the implementation and effectiveness of a longitudinal patient monitoring system designed for adult patients in the unique environment of the Emergency Department (ED).

METHODS

A novel participatory action research (PAR) approach is taken where socio-technical systems (STS) theory and analysis informs the implementation through the improvement methodology of 'Plan Do Study Act' (PDSA) cycles. We hypothesise that conducting an STS analysis of the ED before beginning the PDSA cycles will provide for a much richer understanding of the current situation and possible challenges to implementing the ED-specific longitudinal patient monitoring system. This methodology will enable both a process and an outcome evaluation of implementing the ED-specific longitudinal patient monitoring system. Process evaluations can help distinguish between interventions that have inherent faults and those that are badly executed.

DISCUSSION

Over 1.2 million patients attend EDs annually in Ireland; the successful implementation of an ED-specific longitudinal patient monitoring system has the potential to affect the care of a significant number of such patients. To the best of our knowledge, this is the first study combining PAR, STS and multiple PDSA cycles to evaluate the implementation of an ED-specific longitudinal patient monitoring system and to determine (through process and outcome evaluation) whether this system can significantly improve patient outcomes by early detection and appropriate intervention for patients at risk of clinical deterioration.