Developing medical simulations for opioid overdose response training: A qualitative analysis of narratives from responders to overdoses

Medical simulation offers a controlled environment for studying challenging clinical care situations that are difficult to observe directly. Overdose education and naloxone distribution (OEND) programs aim to train potential rescuers in responding to opioid overdoses, but assessing rescuer performance in real-life situations before emergency medical services arrive is exceedingly complex. There is an opportunity to incorporate individuals with firsthand experience in treating out-of-hospital overdoses into the development of simulation scenarios. Realistic overdose simulations could provide OEND programs with valuable tools to effectively teach hands-on skills and support context-sensitive training regimens. In this research, semi-structured interviews were conducted with 17 individuals experienced in responding to opioid overdoses including emergency department physicians, first responders, OEND program instructors, and peer recovery specialists. Two coders conducted qualitative content analysis using open and axial thematic coding to identify nuances associated with illicit and prescription opioid overdoses. The results are presented as narrative findings complemented by summaries of the frequency of themes across the interviews. Over 20 hours of audio recording were transcribed verbatim and then coded. During the open and axial thematic coding process several primary themes, along with subthemes, were identified, highlighting the distinctions between illicit and prescription opioid overdoses. Distinct contextual details, such as locations, clinical presentations, the environment surrounding the patient, and bystanders' behavior, were used to create four example simulations of out-of-hospital overdoses. The narrative findings in this qualitative study offer context-sensitive information for developing out-of-hospital overdose scenarios applicable to simulation training. These insights can serve as a valuable resource, aiding instructors and researchers in systematically creating evidence-based scenarios for both training and research purposes.

Evaluating rescuer performance in response to opioid overdose in a community setting: Evidence for medically appropriate process measures

Overdose education and naloxone distribution (OEND) programs are widely accepted to reduce opioid overdose deaths. However, there is currently no validated instrument to evaluate the skills of learners completing these programs. Such an instrument could provide feedback to OEND instructors and allow researchers to compare different educational curricula. The aim of this study was to identify medically appropriate process measures with which to populate a simulation-based evaluation tool. Researchers conducted interviews with 17 content experts, including healthcare providers and OEND instructors from south-central Appalachia, to collect detailed descriptions of the skills taught in OEND programs. Researchers used three cycles of open coding, thematic analysis, and consulted currently available medical guidelines to identify thematic occurrences in qualitative data. There was consensus among content experts that the appropriate nature and sequence of potentially lifesaving actions during an opioid overdose is dependent on clinical presentation. Isolated respiratory depression requires a distinct response compared to opioid-associated cardiac arrest. To accommodate these different clinical presentations, raters populated an evaluation instrument with the detailed descriptions of overdose response skills, such as naloxone administration, rescue breathing, and chest compressions. Detailed descriptions of skills are essential to the development of an accurate and reliable scoring instrument. Furthermore, evaluation instruments, such as the one developed from this study, require a comprehensive validity argument. In future work, the authors will integrate the evaluation instrument in high-fidelity simulations, which are safe and controlled environments to study trainees' application of hands-on skills, and conduct formative assessments.

The DoCTRINE Guidelines: Defined Criteria To Report INnovations in Education

PURPOSE

Reporting guidelines assist authors in conducting and describing their research in alignment with evidence-based and expert-determined standards. However, published research-oriented guidelines do not capture all of the components that must be present in descriptions of educational innovations in health professions education. The authors aimed to create guidelines for educational innovations in curriculum development that would be easy for early-career educators to use, support reporting necessary details, and promote educational scholarship.

METHOD

Beginning in 2017, the authors systematically developed a reporting checklist for educational innovations in curriculum development, called Defined Criteria To Report INnovations in Education (DoCTRINE), and collected validity evidence for its use according to the 4 inferences of Kane's framework. They derived the items using a modified Delphi method, followed by pilot testing, cognitive interviewing, and interrater reliability testing. In May-November 2019, they implemented DoCTRINE for authors submitting to MedEdPORTAL, half of whom were randomized to receive the checklist (intervention group). The authors scored manuscripts using DoCTRINE while blinded to group assignment, and they collected data on final editorial decisions.

RESULTS

The final DoCTRINE checklist consists of 19 items, categorized into 5 components: introduction, curriculum development, curriculum implementation, results, and discussion. The overall interrater agreement was 0.91. Among the 108 manuscripts submitted to MedEdPORTAL during the study period, the mean (SD) total score was higher for accepted than rejected submissions (16.9 [1.73] vs 15.7 [2.24], P = .006). There were no significant differences in DoCTRINE scores between the intervention group, who received the checklist, and the control group, who did not.

CONCLUSIONS

The authors developed DoCTRINE, using systematic approaches, for the scholarly reporting of educational innovations in curriculum development. This checklist may be a useful tool for supporting the publishing efforts of early-career faculty.

A review of performance assessment tools for rescuer response in opioid overdose simulations and training programs

Since the 1990s, more than 600 overdose response training and education programs have been implemented to train participants to respond to an opioid overdose in the United States. Given this substantial investment in overdose response training, valid assessment of a potential rescuers' proficiency in responding to an opioid overdose is important. The aim of this article is to review the current state of the literature on outcome measures utilized in opioid overdose response training. Thirty-one articles published between 2014 and 2020 met inclusion criteria. The reviewed articles targeted laypersons, healthcare providers, and first responders. The assessment tools included five validated questionnaires, fifteen non-validated questionnaires, and nine non-validated simulation-based checklists (e.g., completion of critical tasks and time to completion). Validated multiple choice knowledge assessment tools were commonly used to assess the outcomes of training programs. It is unknown how scores on these assessment tools may correlate with actual rescuer performance responding to an overdose. Seven studies reported ceiling effects most likely attributed to participants' background medical knowledge or experience. The inclusion of simulation-based outcome measures of performance, including the commission of critical errors and the time to naloxone administration, provides better insight into rescuer skill proficiency.