Development of a Simulation-Based Mastery Learning Curriculum for Breaking Bad News

INTRODUCTION

Physician communication impacts patient outcomes. However, communication skills, especially around difficult conversations, remain suboptimal, and there is no clear way to determine the validity of entrustment decisions. The aims of this study were to 1) describe the development of a simulation-based mastery learning (SBML) curriculum for breaking bad news (BBN) conversation skills and 2) set a defensible minimum passing standard (MPS) to ensure uniform skill acquisition among learners.

INNOVATION

An SBML BBN curriculum was developed for fourth-year medical students. An assessment tool was created to evaluate the acquisition of skills involved in a BBN conversation. Pilot testing was completed to confirm improvement in skill acquisition and set the MPS.

OUTCOMES

A BBN assessment tool containing a 15-item checklist and six scaled items was developed. Students' checklist performance improved significantly at post-test compared to baseline (mean 65.33%, SD = 12.09% vs mean 88.67%, SD = 9.45%, P < 0.001). Students were also significantly more likely to have at least a score of 4 (on a five-point scale) for the six scaled questions at post-test. The MPS was set at 80%, requiring a score of 12 items on the checklist and at least 4 of 5 for each scaled item. Using the MPS, 30% of students would require additional training after post-testing.

COMMENTS

We developed a SBML curriculum with a comprehensive assessment of BBN skills and a defensible competency standard. Future efforts will expand the mastery model to larger cohorts and assess the impact of rigorous education on patient care outcomes.

Preparedness and education in airway management

A prepared airway practitioner performs safe airway management, displaying skill, knowledge, and a full awareness of human factors, within a culture of safety. The education of prepared practitioners should include deliberate practice and distributed learning and should aim for expertise rather than mere competence. Translational outcomes from improved education and training can significantly decrease patient morbidity and mortality.

The impact of simulation-based mastery learning, booster session timing and clinical exposure on confidence in intercostal drain insertion: a survey of internal medicine trainees in Scotland

BACKGROUND

Intercostal chest drain (ICD) insertion is a skill that medical trainees lack confidence in performing. This study explores the impact of a national programme of Simulation-Based Mastery Learning (SBML) on procedural confidence, including the impact of time intervals between booster sessions and interim clinical experience.

METHODS

Internal Medicine Trainees in Scotland were surveyed about confidence and clinical experience with ICD insertion before and immediately after SBML and booster session. Data were matched and analysed using paired sample t-tests. Short interval and long interval groups were compared using Student's unpaired t-test. The impact of interim clinical experience was assessed using Analysis of Variance.

RESULTS

Mean confidence in ICD insertion rose following SBML, fell between initial and booster session, and increased again following booster session (P =  < 0.001). 33 of 74 trainees had successfully inserted an ICD between sessions. Fall in confidence was unaffected by the time interval between training sessions, but was mitigated by interim clinical experience.

CONCLUSIONS

SBML boosts trainee confidence in ICD insertion. However, there is evidence of confidence decay, possibly due to a lack of clinical experience between sessions. More research is needed to explore barriers to transfer of skills from simulated to real-world environments.

Use of Simulation-Based Mastery Learning Curriculum to Improve Difficult Conversation Skills Among Anesthesiologists: A Pilot Study

Background

Breaking bad news (BBN) is an important clinical task for physicians. Unfortunately, there is no standard method to teach and assess these skills of anesthesiologists. Although anesthesiology has become a relatively safe medical specialty, complications still occur that require disclosure to patients and their families. Disclosure of bad news can be a significant source of stress for clinicians, especially for those who have low confidence in their BBN skills. Anesthesiologists' skills in BBN can be improved with simulation-based mastery learning (SBML), an intense form of competency-based learning.

Methods

An SBML curriculum was developed using the SPIKES (Situation, Perception, Invitation, Knowledge, Emotion, Summarize) framework for BBN and the NURSE (Naming, Understanding, Respecting, Supporting, Exploring) statements for expressing empathy. A pretest-posttest study was conducted from March 2020 to June 2022 to evaluate anesthesiologists' performance in BBN. Participants completed a 2-hour curriculum consisting of a pretest, didactic session, deliberate practice with feedback, and a posttest. Anesthesiologists were assessed using a 16-item skills checklist.

Results

Six anesthesiology attendings and 14 anesthesiology fellows were enrolled in the study. Three of 20 participants met the minimum passing score (MPS) at the time of their pretest. All study participants met the MPS on their first posttest (P < .001). The median participant confidence in BBN significantly increased (3 to 4, P < .001). Overall course satisfaction in the curriculum was high, with a median score of 5.

Conclusions

Our study demonstrates that a BBN SBML curriculum for anesthesiologists significantly improved communication skills and confidence in a simulated environment. Because only 3 participants met the MPS before training, our results suggest that anesthesiologists could benefit from further education to gain effective communication skills and that SBML training may be effective to achieve this result.

Training as imagined? A critical realist analysis of Scotland's internal medicine simulation programme

BACKGROUND

Evaluating the impact of simulation-based education (SBE) has prioritised demonstrating a causal link to improved patient outcomes. Recent calls herald a move away from looking for causation to understanding 'what else happened'. Inspired by Shorrock's varieties of human work from patient safety literature, this study draws on the concept of work-as-done versus work-as-imagined. Applying this to SBE recognises that some training impacts will be unexpected, and the realities of training will never be quite as imagined. This study takes a critical realist stance to explore the experience and consequences, intended and unintended, of the internal medicine training (IMT) simulation programme in Scotland, to better understand 'training-as-done'.

METHODS

Critical realism accepts that there is a reality to uncover but acknowledges that our knowledge of reality is inevitably our construction and cannot be truly objective. The IMT simulation programme involves three courses over a 3-year period: a 3-day boot camp, a skills day and a 2-day registrar-ready course. Following ethical approval, interviews were conducted with trainees who had completed all courses, as well as faculty and stakeholders both immersed in and distant from course delivery. Interviews were audio-recorded, transcribed verbatim and analysed using critical realist analysis, influenced by Shorrock's proxies for work-as-done.

RESULTS

Between July and December 2023, 24 interviews were conducted with ten trainees, eight faculty members and six stakeholders. Data described proxies for training-as-done within three broad categories: design, experience and impact. Proxies for training design included training-as-prescribed, training-as-desired and training-as-prioritised which compete to produce training-as-standardised. Experience included training-as-anticipated with pre-simulation anxiety and training-as-unintended with the valued opportunity for social comparison as well as a sense of identity and social cohesion. The impact reached beyond the individual trainee with faculty development and inspiration for other training ventures.

CONCLUSION

Our findings highlight unintended consequences of SBE such as social comparison and feeling 'valued as a trainee, valued as a person'. It sheds light on the fear of simulation, reinforcing the importance of psychological safety. A critical realist approach illuminated the 'bigger picture', revealing insights and underlying mechanisms that allow this study to present a new framework for conceptualising training evaluation.

Development of a Simulation-Based Mastery Learning Curriculum for Late Goals of Care Discussions

INTRODUCTION

Fellows in critical care medicine (CCM) routinely help patients and families navigate complex decisions near the end of life. These "late goals of care" (LGOC) discussions require rigorous skills training and impact patient care. Innovation is needed to ensure that fellow training in leading these discussions is centered on reproducible competency-based standards. The aims of this study were to (1) describe the development of a simulation-based mastery learning (SBML) curriculum for LGOC discussions and (2) set a defensible minimum passing standard (MPS) to ensure uniform skill acquisition among learners.

INNOVATION

We developed an SBML curriculum for CCM fellows structured around REMAP, a mnemonic outlining foundational components of effective communication around serious illness. A multidisciplinary expert panel iteratively created an LGOC discussion assessment tool. Pilot testing was completed to refine the checklist, set the MPS, and assess skill acquisition.

OUTCOMES

The LGOC discussion assessment tool included an 18-item checklist and 6 scaled items. The tool produced reliable data (k ≥ 0.7 and ICC of ≥ 0.7). Using the Mastery Angoff method, the panel set the MPS at 87%. Ten CCM fellows participated in the pilot study. Performance on the checklist significantly improved from a median score of 52% (IQR 44%-72%) at pretest to 96% (IQR 82%-97%) at post-test (P = 0.005). The number of learners who met the MPS similarly improved from 10% during pre-testing to 70% during post-testing (P = 0.02).

COMMENT

We describe the development of a LGOC SBML curriculum for CCM fellows which includes a robust communication skills assessment and the delineation of a defensible MPS.