Mock Pages Are a Valid Construct for Assessment of Clinical Decision Making and Interprofessional Communication

Assessing readiness: the impact of an experiential learning entrustable professional activity-based residency preparatory course

As medical schools move to integrate the Core Entrustable Professional Activities for Entering Residency (EPAs) into curricula and address the transition from student to resident, residency preparatory courses have become more prevalent. The authors developed an experiential learning EPA-based capstone course for assessment to determine impact on learner self-assessed ratings of readiness for residency and acquisition of medical knowledge. All fourth-year students from the classes of 2018-2020 completed a required course in the spring for assessment of multiple EPAs, including managing core complaints, performing basic procedures, obtaining informed consent, and providing patient handoffs. Learners selected between three specialty-based parallel tracks - adult medicine, surgery, or pediatrics. Students completed a retrospective pre-post questionnaire to provide self-assessed ratings of residency preparedness and comfort in performing EPAs. Finally, the authors studied the impact of the course on knowledge acquisition by comparing student performance in the adult medicine track on multiple choice pre- and post-tests. Four hundred and eighty-one students were eligible for the study and 452 (94%) completed the questionnaire. For all three tracks, there was a statistically significant change in learner self-assessed ratings of preparedness for residency from pre- to post-course (moderately or very prepared: adult medicine 61.4% to 88.6% [p-value < 0.001]; surgery 56.8% to 81.1% [p-value < 0.001]; pediatrics 32.6% to 83.7% [p-value 0.02]). A similar change was noted in all tracks in learner self-assessed ratings of comfort from pre- to post-course for all studied EPAs. Of the 203 students who participated in the adult medicine track from 2019-2020, 200 (99%) completed both the pre- and post-test knowledge assessments. The mean performance improved from 65.0% to 77.5% (p-value < 0.001). An experiential capstone course for the assessment of EPAs can be effective to improve learner self-assessed ratings of readiness for residency training and acquisition of medical knowledge.

More is not better: A scoping review of simulation in transition to residency programs

BACKGROUND

Transition to residency programs frequently use simulation to promote clinical skills but place limited emphasis on non-clinical skills. We conducted a scoping review to determine how simulation is being used in transition to residency programs and the key non-clinical skills addressed by simulation activities and tools in these programs.

METHODS

We searched PubMed, Scopus, and Embase to identify articles addressing transition to residency, simulation, and non-clinical skills/attributes. Two authors independently screened all abstracts and full-text articles and identified non-clinical attributes elicited in each study. Using descriptive statistics, we characterized the simulation activities and tools and the number and type of non-clinical attributes captured in the programs. Using analysis of variance, we compared the number of non-clinical attributes elicited based on the number of simulation activities used and compared the number of non-clinical attributes elicited based on the number of simulation tools used.

RESULTS

We identified 38 articles that met the study criteria. We characterized simulation activities as mock paging (37%), case-based scenarios (74%), and/or procedural skills training (39%). We found that the most common simulation tools were standardized patients (64.8%), and the most elicited non-clinical attributes were communication skills, critical thinking, and teamwork. Using more simulation activity categories or simulation tools did not increase the number of non-clinical skills elicited.

CONCLUSION

Simulation is used broadly in transition to residency programs but provides training in a few of the non-clinical skills required for a successful transition. Incorporating more simulation activities or tools does not increase the number of non-clinical attributes elicited, illustrating the importance of developing more targeted simulation activities to promote non-clinical skills more effectively.

Confidence-building in a collaborative, multidisciplinary mock page activity for fourth-year medical students

Graduating medical students require multiple skills that are not traditionally taught in medical school. Mock paging activities, where school of medicine (SOM) students receive pages as if from nursing staff in a practice environment, are being used as a tool to teach communication and to enhance confidence. Prior small-scale, specialty specific mock page activities have demonstrated improvement in confidence, communication with other healthcare personnel, and medical decision-making.^3-5 This Mock Page Activity aimed to evaluate the effect on confidence and communication for large graduating SOM class entering different specialties. SOM faculty collaborated with the School of Nursing (SON) faculty in the design and implementation of this activity. Medical students completed pre-/post surveys regarding confidence in communication, interaction with other healthcare professionals, and patient management. Two years of medical student survey data (n=420) after the Mock Page Activity implementation demonstrated a significant increase in general confidence (30.4%, p <0.001) related to receiving communication from nursing staff, making clinical decisions over the phone, and using a pager as a communication device. This multidisciplinary Mock Page Activity improved communication and confidence during paging activities for a large group of medical students pursuing different specialties. Strengths of the study include being the largest, specialty non-specific mock page activity reported in the literature. Limitations include varied individual experience of students and questionable benefits for students pursuing non-patient care careers in medicine. Future directions include iterative improvement based on feedback and incorporation of an interprofessional debrief session to ensure equal benefit to the participating SON students.

Feedback Methods in an Interprofessional Mock Paging Program

BACKGROUND

Many medical schools offer a culminating internship readiness experience. Curricula focus on particular knowledge and skills critical to internship, such as answering urgent nursing pages. Studies have shown student performance improvement with mock paging education programs, but the role of feedback versus self-regulated practice has not been studied.

DESIGN AND METHODS

The interprofessional mock paging program included 156 medical students enrolled in a 4^th-year internship readiness course and 44 master's level direct entry nursing students. Medical students were randomized to receive verbal feedback immediately after each of the three phone calls (intervention group) or delayed written feedback (control group) after the third phone call only. Specialty-specific case scenarios were developed and a single checklist for all scenarios was developed using the communication tool ISBAR. Medical students and nursing students had separate training sessions before the pages commenced. The nursing students administered the phone calls and evaluated the medical students by ISBAR checklist. An interrater reliability measure was obtained with physician observation of a selection of phone calls.

RESULTS

After adjusting for the case effects (different case scenarios for different specialties), students showed no statistically significant differences on checklist scores for case 1 (first case, F = 1.491, df = 1, p = .224), but did show statistically significant differences on checklist scores for case 3 (final case, F = 12.238, df = 1, p = .001). Strong interrater reliability was found between the faculty physician and observed nursing students (ICC = .89).

CONCLUSIONS

Immediate feedback significantly improves student checklist scores with a mock paging program. This finding suggests that coaching with feedback may have advantages above self-regulated learning.

The Development of a Video Intervention to Improve Senior Medical Students' Performance on Outpatient Telephone Encounters: a Delphi Analysis and Randomized Controlled Trial

Introduction

Postgraduate trainees address outpatient telephone calls (OTCs) with little prior training. This study determines the skills necessary for OTCs and examines whether a video intervention improves medical students' performance on simulated OTCs.

Materials and Methods

We utilized a Delphi technique to determine skills needed for OTCs and created a 9-min video teaching these skills. Senior medical students were randomized to Intervention (viewed video) and Control (did not view video) groups. Students were assessed pre-/post-intervention on simulated OTCs. The primary outcome was the between-group difference in improvement.

Results

The Delphi yielded 34 important skills with the highest focus on communication (n = 13) and triage (n = 6). Seventy-two students completed assessments (Control, n = 41; Intervention, n = 31). The score (mean ± SD) improved 4.3% in the Control group (62.3 ± 14.3% to 66.6 ± 25.0%) and 12.2% in the Intervention group (60.7 ± 15.2% to 72.9 ± 20.4%, p = 0.15). The effect size measured by Cohen's d was 0.55, considered effective (> 0.33) for an educational intervention.

Conclusions

This project fills a gap in OTC training. The use of the Delphi technique, intervention development based on the results, and evaluation of efficacy is a process that could be reproduced for other educational gaps.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40670-021-01331-w.

Interprofessional communication (IPC) for medical students: a scoping review

BACKGROUND

Effective Interprofessional Communication (IPC) between healthcare professionals enhances teamwork and improves patient care. Yet IPC training remains poorly structured in medical schools. To address this gap, a scoping review is proposed to study current IPC training approaches in medical schools.

METHODS

Krishna's Systematic Evidence Based Approach (SEBA) was used to guide a scoping review of IPC training for medical students published between 1 January 2000 to 31 December 2018 in PubMed, ScienceDirect, JSTOR, Google Scholar, ERIC, Embase, Scopus and PsycINFO. The data accrued was independently analysed using thematic and content analysis to enhance the reproducibility and transparency of this SEBA guided review.

RESULTS

17,809 titles and abstracts were found, 250 full-text articles were reviewed and 73 full text articles were included. Directed Content analysis revealed 4 categories corresponding to the levels of the Miller's Pyramid whilst thematic analysis revealed 5 themes including the indications, stages of trainings and evaluations, content, challenges and outcomes of IPC training. Many longitudinal programs were designed around the levels of Miller's Pyramid.

CONCLUSION

IPC training is a stage-wise, competency-based learning process that pivots on a learner-centric spiralled curriculum. Progress from one stage to the next requires attainment of the particular competencies within each stage of the training process. Whilst further studies into the dynamics of IPC interactions, assessment methods and structuring of these programs are required, we forward an evidenced based framework to guide design of future IPC programs.

The journeys of three ASPIRE winning medical schools toward excellence in student assessment

Introduction: ASPIRE Excellence Awards in Student Assessment are offered to medical schools with innovative and comprehensive assessment programmes adjudged by international experts, using evidence-based criteria. The journeys of three ASPIRE-winning medical schools toward "assessment excellence" are presented. These schools include Aga Khan University Medical College (AKU-MC), Pakistan, Southern Illinois University School of Medicine (SIUSOM), USA, and University of Leeds School of Medicine, UK. Methods: The unfolding journeys highlighting achievements, innovations, and essential components of each assessment programme were compared to identify differences and commonalities. Results: Cultural contextual differences included developed-versus-developing country, east-west, type of regulatory bodies, and institutional-versus-national certifying/licensing examinations, which influence curricula and assessments. In all, 12 essential commonalities were found: alignment with institutional vision; sustained assessment leadership; stakeholder engagement; communication between curriculum and assessment; assessment-for-learning and feedback; longitudinal student profiling of outcome achievement; assessment rigor and robustness; 360° feedback from-and-to assessment; continuous enrichment through rigorous quality assurance; societal sensitivity; influencing others; and a "wow factor." Conclusions: Although the journeys of the three medical schools were undertaken in different cultural contexts, similar core components highlight strong foundations in student assessment. The journeys continue as assessment programmes remain dynamic and measurement science expands. This article may be helpful to other institutions pursuing excellence in assessment.

Attitudes Toward Communication and Collaboration After Participation in a Mock Page Program: A Pilot of an Interprofessional Approach to Surgical Residency Preparation

OBJECTIVE

A source of insecurity among new physicians and new nurses is communicating and managing acute changes in patient condition. Mock page programs are an effective method for teaching communication and clinical decision-making skills to medical students. Joint participation in a mock page program provides a unique opportunity for medical and nursing students to practice communication, collaboration, and clinical decision-making in a low-risk learning environment. The purpose of this pilot study was to examine attitudes toward communication and collaboration among medical and nursing students after participation in a mock page program.

DESIGN

Twenty-four medical students participating in a surgical residency preparatory course were paired with 24 senior level nursing students. Over a period of 3 weeks, nursing students delivered 5 mock pages to the medical students. Mock pages consisted of peer-reviewed, prescripted scenarios depicting patients experiencing common postoperative complications. A 12-question survey measuring attitudes toward communication and collaboration between physicians and nurses was administered prior to delivery of the first page and after delivery of the last page. A dependent sample t test was used to perform item analysis.

RESULTS

Individual item analysis of medical student surveys indicated significant improvement in attitudes toward communication and collaboration with nurses in 12 out of 12 survey items (p < 0.05). Individual item analysis of nursing student surveys indicated significant improvement in attitudes toward communication and collaboration with physicians in 6 out of 10 survey items (p < 0.05). There was no decline in attitudes among any of the survey items.

CONCLUSIONS

The results from this study suggest that an interprofessional mock page program improves attitudes toward communication and collaboration between medical and nursing students. This program has the potential to improve the quality of education and increase confidence among medical and nursing students as they prepare for their professional roles.

The intersection of professionalism and interprofessional care: development and initial testing of the interprofessional professionalism assessment (IPA)

Valid assessment of interprofessional education and collaborative practice (IPECP) is challenging. The number of instruments that measure various aspects of IPECP, or in various sites is growing, however. The Interprofessional Professionalism Assessment (IPA) measures observable behaviors of health care professionals-in-training that demonstrate professionalism and collaboration when working with other health care providers in the context of people-centered care. The IPA instrument was created by the Interprofessional Professionalism Collaborative (IPC), a national group representing 12 entry-level health professions and one medical education assessment organization. The instrument was created and evaluated over several years through a comprehensive, multi-phasic process: 1) development of construct and observable behaviors, 2) instrument design, expert review and cognitive interviews, and 3) psychometric testing. The IPA contains 26 items representing six domains of professionalism (altruism and caring, excellence, ethics, respect, communication, accountability), and was tested by 233 preceptors rating health profession learners in the final year of their practical training. These preceptors represented 30 different academic institutions across the U.S., worked in various types of practice sites, and evaluated learners representing 10 different entry-level health professions. Exploratory factor analysis suggested four factors (communication, respect, excellence, altruism and caring) using 21 items with the least amount of missing data, and confirmed, for the most part, a priori expectations. Internal consistency reliability coefficients for the entire instrument and its four subscales were high (all greater than 0.9). Psychometric results demonstrate aspects of the IPA's reliability and validity and its use across multiple health professions and in various practice sites.

Mock Paging and Consult Curriculum to Prepare Fourth-Year Medical Students for Medical Internship

INTRODUCTION

Internship preparation should include curricula to hone key skills such as acute medical management and communication with consulting and interprofessional providers.

METHODS

To enhance these skills, we developed an interprofessional mock paging and consult curriculum incorporating direct observation and peer, faculty, and nursing feedback for fourth-year medical students entering medical internships. Our brief mock paging and consult curriculum was designed as part of a larger 2-week internship preparation course. Our curriculum was delivered in two 2-hour sessions by physician and nurse educators. Sessions were conducted in small groups, offering the opportunity for direct observation and feedback from faculty, nurse educators, and peers. Our curriculum was expanded from a pilot for 10-15 students to 60 students after 2 years of a successful pilot.

RESULTS

Mock paging and consult sessions were highly rated by medical students and resulted in significantly enhanced self-assessment of preparedness in key intern skills such as returning pages, interprofessional communication, calling a consult, and managing acute issues for cross-cover patients.

DISCUSSION

We have demonstrated the effectiveness of a brief, interprofessional mock paging and consult curriculum incorporating faculty, nurse educator, and peer feedback. The tenets of our curriculum can be widely adopted for other learner groups.