What are the measures that can be used to assess performance during in situ Paediatric Emergency Medicine Simulation?

Evaluation of a multiple-encounter in situ simulation for orientation of staff to a new paediatric emergency service: a single-group pretest/post-test study

AIM

To assess the utility of a multiple-encounter in-situ (MEIS) simulation as an orientation tool for multidisciplinary staff prior to opening a new paediatric emergency service.

METHODS

A single-group pretest/post-test study was conducted. During the MEIS simulation, multidisciplinary staff with participant or observer roles managed eight children (mannequins) who attended triage with their parent/guardians (clinical facilitators) for a range of emergency presentations (structured scenarios designed to represent the expected range of presentations plus test various clinical pathways/systems). Participants were debriefed to explore clinical, systems and crisis-resource management issues. Participants also completed a pre-intervention and post-intervention questionnaire comprising statements about role confidence and orientation adequacy. Pre-test and post-test results were analysed using t-test and Wilcoxon signed rank test.

RESULTS

Eighty-nine staff participated in the MEIS simulation, with the majority completing the pre-simulation and post-simulation questionnaire. There was a significant improvement in post-intervention versus pre-intervention Likert scores for role confidence and orientation adequacy (p=0.001 and <0.001, respectively); effect sizes suggested the greatest impact was on orientation adequacy. Nearly all scenarios resulted in significant increases in participants' confidence levels.

CONCLUSIONS

The MEIS simulation was of utility in orientation of staff, at least with respect to self-reported role confidence and orientation adequacy. Its effectiveness in practice or compared with other orientation techniques was not assessed, but it did identify several flaws in planned systems allowing remediation prior to opening.