Teaching the tutors: use of an OSTE to train medical students to be peer tutors

First-year medical students are often challenged by the rapid pace and large volume of content that must be learned. Peer teaching has emerged as a supportive educational strategy. However, the most effective strategies for training peer tutors (PTs) for their role are not known. This paper examines the use of an Objective Structured Teaching Exercise (OSTE) to augment PT training sessions. Applying deliberate practice as a conceptual framework, an OSTE was used to provide tutors with an opportunity to practice their skills and receive feedback about their performance when meeting with a student presenting with a challenge. The newly trained PTs were required to assess a standardized student, determine challenge(s) being experienced, and present options to address the challenge(s). Standardized students evaluated the tutors' performance and a pre- and post-OSTE questionnaire was used to determine whether the OSTE was effective in increasing the confidence level of PTs to effectively assess and support students seeking help. Participants reported an increase in confidence in their ability to assess areas requiring improvement, understand the active learning strategies, and suggest appropriate active learning study strategies. Evaluations completed by standardized students documented that newly trained PTs accurately diagnosed the challenge presented in the OSTE and in most cases PTs asked all relevant questions to assess. Increased self-efficacy promotes PT's capacity to perform their work and feedback during an OSTE can further advance required skills. Aggregate OSTE results can also inform efforts to enhance the PT training program.NEW & NOTEWORTHY This novel application of the Objective Structured Teaching Exercise (OSTE) was done to enhance tutors' skills as valued members of our integrated academic support program. The OSTE provided feedback to the tutors and enabled us to identify a need for enhanced tutor training in active learning strategies. The OSTE can be adapted for use in other health science educational programs to enhance their training programs and to assess tutor's skills in preparation for their role.

'We're like Spider-Man; with great power comes great responsibility': Coaches' experiences supporting struggling medical students

BACKGROUND

Medical students can experience a range of academic and non-academic struggles. Coaching is a valuable strategy to support learners, but coaches describe working with struggling learners as taxing. Transformative learning theory (TLT) provides insights into how educators grow from challenging experiences to build resilience. This study explores how coaches evolve as educators through supporting struggling students.

METHODS

This qualitative study grounded in an interpretivist paradigm used interviews of longitudinal medical student coaches at two academic institutions. Interviews, using TLT as a sensitizing concept, explored coaches' experience coaching struggling learners. We performed thematic analysis.

RESULTS

We interviewed 15 coaches. Coaches described supporting students through multi-faceted struggles which often surprised the coach. Three themes characterized coaches' experiences: personal responsibility, emotional response, and personal learning. Coaches shouldered high personal responsibility for learners' success. For some, this burden felt emotional, raised parental instincts and questions about maintaining boundaries with learners. Coaches evolved their coaching approach, challenged biases, and built skills. Coaches learned to better appreciate the learner point of view and employ resources to support students.

DISCUSSION

Through navigating learner struggles, educators can gain self-efficacy, learn to understand learners' perspectives, and evolve their coaching approach to lessen their personal emotional burden through time.

A novel virtual course to teach medical students high-value decision-making

BACKGROUND

Although a clinician's ability to employ high-value decision-making is influenced by training, many undergraduate medical education programmes lack a formal curriculum in high-value, cost-conscious care. We present a curriculum developed through a cross-institutional collaboration that was used to teach students at two institutions about this topic and can serve as a framework for other institutions to develop similar curricula.

APPROACH

The faculty from the University of Virginia and the Johns Hopkins University School of Medicine created a 2-week-long online course to teach medical students the fundamentals of high-value care. The course consisted of learning modules, clinical cases, textbook studies, journal clubs and a competitive 'Shark Tank' final project where students proposed a realistic intervention to promote high-value clinical care.

EVALUATION

Over two-thirds of students rated the course's quality as excellent or very good. Most found the online modules (92%), assigned textbook readings (89%) and 'Shark Tank' competition (83%) useful. To evaluate the student's ability to apply the concepts learned during the course in clinical contexts, we developed a scoring rubric based on the New World Kirkpatrick Model to evaluate students' proposals. Groups chosen as finalists (as determined by faculty judges) were more likely to be fourth-year students (56%), achieved higher overall scores (p = 0.03), better incorporated cost impact at several levels (patient, hospital and national) (p = 0.001) and discussed both positive and negative impacts on patient safety (p = 0.04).

IMPLICATIONS

This course provides a framework for medical schools to use in their teaching of high-value care. Cross-institutional collaboration and online content overcame local barriers such as contextual factors and lack of faculty expertise, allowed for greater flexibility, and enabled focused curricular time to be spent on a capstone project competition. Prior clinical experience amongst medical students may be an enabling factor in promoting application of learning related to high-value care.

Becoming by doing: How women in academic health sciences build a leadership identity through project experiences

PURPOSE

Leadership development programs often use institutional projects to activate learning. We explored how project work shaped leadership identity formation in senior women leaders from one academic health science center who enrolled in The Hedwig von Ameringen Executive Leadership in Academic Medicine (ELAM^®) program.

MATERIALS AND METHODS

We interviewed ELAM Fellows and conducted a qualitative analysis of transcripts. Our primary analysis focused on the influences of projects on Fellows and institutions. Leadership identity formation emerged as a distinct pattern, so this narrative content was separated for secondary analysis. All authors approved the final assignment of themes and codes.

RESULTS

Participants described a multi-dimensional process for developing a leadership identity. Themes encompassed participants' View of Self and One's Image as a Leader, Interpersonal Relationships, and Commitment to a Value-based Goal. These internal factors grounded external influences, such as interactions with colleagues and institutional leaders, and the world beyond the institution.

CONCLUSIONS

We examined the process of leadership identity formation from the perspective of women leaders in academic health sciences who completed an institutional project during a leadership development program. Findings illustrate how internal and external forces, experienced in the context of project work, combine to influence leadership identity formation in women.

Fidelity to Best Practices in EPA Implementation: Outcomes Supporting Use of the Core Components Framework From the University of Virginia Entrustable Professional Activity Program

PROBLEM

The rapid expansion of entrustable professional activity (EPA) assessment programs has led to calls to ensure fidelity in implementation and integrity in meeting the goals of competency-based medical education. Initiated in July 2017, in advance of the articulated core components of EPA implementation, this article describes the structure and outcomes of the University of Virginia (UVA) EPA Program and provides support for the identified essential components.

APPROACH

The UVA EPA Program includes workplace assessments by residents/fellows, attending faculty, and master assessors (MAs), experienced clinicians who assess students across disciplines and clinical settings. All assessors participate in formal professional development and provide verbal and written comments to support their supervision ratings. The Entrustment Committee, composed of 12 MAs, uses a shared mental model and aggregates all assessor data to make a high-stakes summative entrustment decision about students' readiness to assume the role of an acting intern.

OUTCOMES

Since 2017, over 2,000 assessors have completed 56,969 EPA assessments for 1,479 students. Ninety-four percent of assessments have been done during the clerkship phase. Residents/fellows have completed a mean of 18 assessments, attending faculty a mean of 27, and MAs a mean of 882. Seventy-four percent of observed encounters involved patients with acute concerns with or without a co-morbid condition. Fifty percent of assessments occurred in inpatient and 32% in ambulatory settings. Eighty-seven percent of assessments contained narrative comments with more than 100 characters.

NEXT STEPS

Planned next steps will include earlier identification of students who require individualized learning to promote the development of skills related to EPAs, expansion of the remediation program to enable more students to engage in a clinical performance mastery elective, and creation of targeted professional development for assessors to reinforce the tenets of the EPA program.

The use of EPA assessments in decision-making: Do supervision ratings correlate with other measures of clinical performance?

BACKGROUND

Entrustable professional activities (EPAs) have been introduced as a framework for teaching and assessment in competency-based educational programs. With growing use, has come a call to examine the validity of EPA assessments. We sought to explore the correlation of EPA assessments with other clinical performance measures to support use of supervision ratings in decisions about medical students' curricular progression.

METHODS

Spearman rank coefficients were used to determine correlation of supervision ratings from EPA assessments with scores on clerkship evaluations and performance on an end-of-clerkship-year Objective Structured Clinical Examination (CPX).

RESULTS

Both overall clinical evaluation items score (rho 0.40; n = 166) and CPX patient encounter domain score (rho 0.31; n = 149) showed significant correlation with students' overall mean EPA supervision rating during the clerkship year. There was significant correlation between mean supervision rating for EPA assessments of history, exam, note, and oral presentation skills with scores for these skills on clerkship evaluations; less so on the CPX.

CONCLUSIONS

Correlation of EPA supervision ratings with commonly used clinical performance measures offers support for their use in undergraduate medical education. Data supporting the validity of EPA assessments promotes stakeholders' acceptance of their use in summative decisions about students' readiness for increased patient care responsibility.

Implementing Systematic Faculty Development to Support an EPA-Based Program of Assessment: Strategies, Outcomes, and Lessons Learned

Problem: Development of a novel, competency-based program of assessment requires creation of a plan to measure the processes that enable successful implementation. The principles of implementation science outline the importance of considering key drivers that support and sustain transformative change within an educational program. The introduction of Entrustable Professional Activities (EPAs) as a framework for assessment has underscored the need to create a structured plan to prepare assessors to engage in a new paradigm of assessment. Although approaches to rater training for workplace-based assessments have been described, specific strategies to prepare assessors to apply standards related to the level of supervision a student needs have not been documented. Intervention: We describe our systematic approach to prepare assessors, faculty and postgraduate trainees, to complete EPA assessments for medical students during the clerkship phase of our curriculum. This institution-wide program is designed to build assessors' skills in direct observation of learners during authentic patient encounters. Assessors apply new knowledge and practice skills in using established performance expectations to determine the level of supervision a learner needs to perform clinical tasks. Assessors also learn to provide feedback and narrative comments to coach students and promote their ongoing clinical development. Data visualizations for assessors facilitate reinforcement of the tenets learned during training. Collaborative learning and peer feedback during faculty development sessions promote the formation of a community of practice among assessors. Context: Faculty development for assessors was implemented in advance of implementation of the EPA program. Assessors in the program include residents/fellows who work closely with students, faculty with discipline-specific expertise and a group of experienced clinicians who were selected to serve as experts in competency-based EPA assessments, the Master Assessors. Training focused on creating a shared understanding about the application of criteria used to evaluate student performance. EPA assessments based on the AAMC's Core Entrustable Professional Activities for Entering Residency, were completed in nine core clerkships. EPA assessments included a supervision rating based on a modified scale for use in undergraduate medical education. Impact: Data from EPA assessments completed during the first year of the program were analyzed to evaluate the effectiveness of the faculty development activities implemented to prepare assessors to consistently apply standards for assessment. A systematic approach to training and attention to critical drivers that enabled institution-wide implementation, led to consistency in the supervision rating for students' first EPA assessment completed by any type of assessor, ratings by assessors done within a specific clinical context, and ratings assigned by a group of specific assessors across clinical settings. Lessons learned: A systematic approach to faculty development with a willingness to be flexible and reach potential participants using existing infrastructure, can facilitate assessors' engagement in a new culture of assessment. Interaction among participants during training sessions not only promotes learning but also contributes to community building. A leadership group responsible to oversee faculty development can ensure that the needs of stakeholders are addressed and that a change in assessment culture is sustained.

How Do Clerkship Students Use EPA Data? Illuminating Students' Perspectives as Partners in Programs of Assessment

INTRODUCTION

The implementation of programs of assessment based on Entrustable Professional Activities (EPAs) offers an opportunity for students to obtain unique data to guide their ongoing learning and development. Although authors have explored factors that contribute to trust-based decisions, learners' use of assessors' decisions about the level of supervision they need has not been fully investigated.

METHODS

In this study, we conducted semi-structured interviews of clerkship students who participated in the first year of our EPA program to determine how they interpret and use supervision ratings provided in EPA assessments. Content analysis was performed using concept-driven and open coding.

RESULTS

Nine interviews were completed. Twenty-two codes derived from previous work describing factors involved in trust decisions and 12 novel codes were applied to the interview text. Analyses revealed that students focus on written and verbal feedback from assessors more so than on supervision ratings. Axial coding revealed a temporal organization that categorized how students considered the data from EPA assessments. While factors before, during, and after an assessment affected students' use of information, the relationship between the student and the assessor had impact throughout.

CONCLUSIONS

Although students reported varying use of the supervision ratings, their perspectives about how assessors and students interact and/or partner before, during, and after assessments provide insights into the importance of an educational alliance in making a program of assessment meaningful and acceptable to learners.

You can have both: Coaching to promote clinical competency and professional identity formation

Coaching is a critical tool to guide student development of clinical competency and formation of professional identity in medicine, two inextricably linked concepts. Because progress toward clinical competence is linked to thinking, acting and feeling like a physician, a coach's knowledge about a learner's development of clinical skills is essential to promoting the learner's professional identity formation. A longitudinal coaching program provides a foundation for the formation of coach-learner relationships built on trust. Trusting relationships can moderate the risk and vulnerability inherent in a hierarchical medical education system and allow coaching conversations to focus on the promotion of self-regulated learning and fostering skills for life-long learning. Herein, we describe a comprehensive, longitudinal clinical coaching program for medical students designed to support learners' professional identify formation and effectively promote their emerging competence.

Leadership Development Projects: Bidirectional Impact on Faculty and Institutions

INTRODUCTION

The Hedwig von Ameringen Executive Leadership in Academic Medicine program (ELAM) is a national professional development program for women that includes institutional action projects (IAPs). Although benefits of ELAM participation are well documented, the value of the IAPs has not been specifically evaluated. We explored the experience of ELAM Fellows and leaders from one institution to elucidate how institutional factors influence project implementation and outcomes.

METHODS

Fellows and deans participated in semistructured interviews. We analyzed the transcripts qualitatively to develop themes and describe factors that influenced IAP implementation and outcomes. We used the New World Kirkpatrick Model, an updated version of the widely used Kirkpatrick model of educational program evaluation, as a framework to elucidate how participants applied their leadership learning through project work, and to analyze early results of projects that indicated institutional impact.

RESULTS

Project work had bidirectional impact on the fellows in the program and on the institution itself. Project enablers included: focusing projects on institutional priorities, obtaining sustainable support, and navigating institutional complexity. Leading indicators of institutional outcomes included contributions to institutional leadership and culture, and mutual enhancement of the reputation of the fellow and of the institution.

DISCUSSION

By examining enablers and barriers for institutionally based projects conducted in a national leadership development program, we identified the drivers that facilitated application of leadership learning. Leading indicators of project outcomes reflected bidirectional impact on fellows and the institution, demonstrating outcomes at the highest levels of the New World Kirkpatrick Model.

International Medical Graduates in the Pediatric Workforce in the United States

BACKGROUND AND OBJECTIVES

To describe the supply, distribution, and characteristics of international medical graduates (IMGs) in pediatrics who provide patient care in the United States.

METHODS

Cross-sectional study, combining data from the 2019 Physician Masterfile of the American Medical Association and the Educational Commission for Foreign Medical Graduates database.

RESULTS

In total, 92 806 pediatric physicians were identified, comprising 9.4% of the entire US physician workforce. Over half are general pediatricians. IMGs account for 23.2% of all general pediatricians and pediatric subspecialists. Of all IMGs in pediatrics, 22.1% or 4775 are US citizens who obtained their medical degree outside the United States or Canada, and 15.4% (3246) attended medical school in the Caribbean. Fifteen non-US medical schools account for 29.9% of IMGs currently in active practice in pediatrics in the United States. IMGs are less likely to work in group practice or hospital-based practice and are more likely to be employed in solo practice (compared with US medical school graduates).

CONCLUSIONS

With this study, we provide an overview of the pediatric workforce, quantifying the contribution of IMGs. Many IMGs are US citizens who attend medical school abroad and return to the United States for postgraduate training. Several factors, including the number of residency training positions, could affect future numbers of IMGs entering the United States. Longitudinal studies are needed to better understand the implications that workforce composition and distribution may have for the care of pediatric patients.

Scholarly Evaluation of Curricula and Educational Programs: Using a Systematic Approach to Produce Publishable Scholarship

The goal of designing innovative curricula and programs in medical education is to create interventions that will change the attitudes, knowledge, skills, and behaviors of learners in order to prepare them to contribute to the health of patients and communities they serve. Systematic evaluation allows curriculum/program developers to optimize their curriculum/program and ensure that the goals of the curriculum/program are met. While curriculum/program evaluation is distinct from educational research, when approached in a rigorous manner, curriculum/program evaluation can be published as educational scholarship. The goal of our paper is to equip readers with tools to apply a scholarly approach to curriculum/program evaluation to produce findings that will advance the field. We start by defining curricula, educational programs, learner assessment, and evaluation. We then briefly describe examples of best practice models for curriculum/program development that incorporate the critical step of planning for evaluation. Building on published work, we distinguish the use of best practice models and conceptual frameworks to inform curriculum/program development and evaluation. More specifically, we outline steps to plan an evaluation that demonstrates WHAT was effective, describes HOW the curriculum/program contributed to the outcomes achieved, and points to WHY the curriculum/program led to the outcomes observed. We conclude with key considerations for publishing findings of an evaluation, including what to include in each section of a manuscript.

Simulation Training Incorporating a Pulsatile Carotid Endarterectomy Model Results in Increased Procedure-Specific Knowledge, Confidence, and Comfort in Post-graduate Trainees

OBJECTIVES

Surgical simulation has been used to facilitate the acquisition of vascular surgery skills. However, high cost and limited availability may restrict the use of this educational resource. We report how instruction using a low-cost, pulsatile, carotid endarterectomy (CEA) benchtop surgical simulation model can be used to enhance learners' procedure-specific knowledge, comfort, and confidence in performing the steps of a CEA procedure DESIGN: A single instructor engaged each participant in a one-on-one instructional session during which the instructor demonstrated, and then the participants performed, the steps of a CEA. Participants completed a pre- and postintervention assessment of knowledge and attitudes about preforming a CEA and use of simulation as a learning tool. Postintervention, participants rated the impact of the simulation model on their learning. A Related T-test and Wilcoxin signed Rank Test were used to compare pre- and postintervention results.

SETTINGS

University of Virginia Health System, Charlottesville, Virginia.

PARTICIPANTS

Seventeen postgraduate trainees.

RESULTS

A significant difference was observed in pre- and postknowledge scores (48% vs 91% correct, p < 0.01). Trainee confidence (1.65 vs 2.88, p < 0.01) and comfort (1.59 vs 2.82, p < 0.01) with doing the procedure also increased significantly. Sixteen (94%) responded that use of the simulator was extremely or very important as a tool for learning. All 17 trainees (100%) reported that the simulation experience was either essential or very useful in helping them learn how to perform a CEA. Sixty-five percent responded that they were extremely likely to apply the skills learned during the intervention the next time they performed a CEA.

CONCLUSIONS

A low-cost, pulsatile CEA simulation model used as an educational tool increased procedure-specific knowledge, comfort, and confidence among trainees. Learner's increased confidence and affirmation that they are likely to apply the learned skills in a clinical setting support the use of this educational approach to impact trainee behaviors.

Case-Based Teaching: Does the Addition of High-Fidelity Simulation Make a Difference in Medical Students' Clinical Reasoning Skills?

CONTEXT

Situativity theory posits that learning and the development of clinical reasoning skills are grounded in context. In case-based teaching, this context comes from recreating the clinical environment, through emulation, as with manikins, or description. In this study, we sought to understand the difference in student clinical reasoning abilities after facilitated patient case scenarios with or without a manikin.

METHODS

Fourth-year medical students in an internship readiness course were randomized into patient case scenarios without manikin (control group) and with manikin (intervention group) for a chest pain session. The control and intervention groups had identical student-led case progression and faculty debriefing objectives. Clinical reasoning skills were assessed after the session using a 64-question script concordance test (SCT). The test was developed and piloted prior to administration. Hospitalist and emergency medicine faculty responses on the test items served as the expert standard for scoring.

RESULTS

Ninety-six students were randomized to case-based sessions with (n = 48) or without (n = 48) manikin. Ninety students completed the SCT (with manikin n = 45, without manikin n = 45). A statistically significant mean difference on test performance between the two groups was found (t = 3.059, df = 88, p = .003), with the manikin group achieving higher SCT scores.

CONCLUSION

Use of a manikin in simulated patient case discussion significantly improves students' clinical reasoning skills, as measured by SCT. These results suggest that using a manikin to simulate a patient scenario situates learning, thereby enhancing skill development.

How a Year of Clinical Experience Affects Students' Perspectives About the Supervision They Need to Perform Entrustable Professional Activities

Assessment of entrustable professional activities (EPAs) requires a supervisor to determine a learner's need for supervision to perform the tasks. This study examines medical students' perspectives, after completing their core clerkships, about the supervision they need to perform tasks defined by the core EPAs for entering residency. Compared with assertions at the start of the clerkship year, on retrospective self-assessment, students reported needing higher levels of supervision predominantly for activities commonly done during clinical rotations. Students remain confident in their ability to perform tasks with autonomy in the post-clerkship phase, however, raising concern about their abilities to discern limitations and appropriately request supervision.

Moving Toward Summative Competency Assessment to Individualize the Postclerkship Phase

In the move toward competency-based medical education, leaders have called for standardization of learning outcomes and individualization of the learning process. Significant progress has been made in establishing defined expectations for the knowledge, skills, attitudes, and behaviors required for successful transition to residency training, but individualization of educational processes to assist learners in reaching these competencies has been predominantly conceptual to date. The traditional time-based structure of medical education has posed a challenge to individualization within the curriculum and has led to more attention on innovations that facilitate transition from medical school to residency. However, a shift of focus to the clerkship-to-postclerkship transition point in the undergraduate curriculum provides an opportunity to determine how longitudinal competency-based assessments can be used to facilitate intentional and individualized structuring of the long-debated fourth year.This Perspective demonstrates how 2 institutions-the University of Virginia School of Medicine and the University of Michigan Medical School-are using competency assessments and applying standardized outcomes in decisions about individualization of the postclerkship learning process. One institution assesses Core Entrustable Professional Activities for Entering Residency, whereas the other has incorporated Accreditation Council for Graduate Medical Education core competencies and student career interests to determine degrees of flexibility in the postclerkship phase. Individualization in addition to continued assessment of performance presents an opportunity for intentional use of curriculum time to develop each student to be competently prepared for the transition to residency.

Enhancing Educational Scholarship Through Conceptual Frameworks: A Challenge and Roadmap for Medical Educators

Historically, health sciences education has been guided by tradition and teacher preferences rather than by the application of practices supported by rigorous evidence of effectiveness. Although often underutilized, conceptual frameworks-theories that describe the complexities of educational and social phenomenon-are essential foundations for scholarly work in education. Conceptual frameworks provide a lens through which educators can develop research questions, design research studies and educational interventions, assess outcomes, and evaluate the impact of their work. Given this vital role, conceptual frameworks should be considered at the onset of an educational initiative. Use of different conceptual frameworks to address the same topic in medical education may provide distinctive approaches. Exploration of educational issues by employing differing, theory-based approaches advances the field through the identification of the most effective educational methods. Dissemination of sound educational research based on theory is similarly essential to spark future innovation. Ultimately, this rigorous approach to medical education scholarship is necessary to allow us to establish how our educational interventions impact the health and well-being of our patients.

How Mentor Identity Evolves: Findings From a 10-Year Follow-up Study of a National Professional Development Program

PURPOSE

Despite academic medicine's endorsement of professional development and mentoring, little is known about what junior faculty learn about mentoring in implicit curricula of professional development programs, and how their mentor identity evolves in this context. The authors explored what faculty-participants in the Educational Scholars Program implicitly learned about mentoring and how the implicit curriculum affected mentor identity transformation.

METHOD

Semistructured interviews with 19 of 36 former faculty-participants were conducted in 2016. Consistent with constructivist grounded theory, data collection and analysis overlapped. The authors created initial codes informed by Ibarra's model for identity transformation, iteratively revised codes based on incoming data patterns, and created visual representations of relationships amongst codes to gain a holistic, shared understanding of the data.

RESULTS

In the implicit curriculum, faculty-participants learned the importance of having multiple mentors, the value of peer mentors, and the incremental process of becoming a mentor. The authors used Ibarra's model to understand how the implicit curriculum worked to transform mentor identity: Faculty-participants reported observing mentors, experimenting with different ways to mentor and to be a mentor, and evaluating themselves as mentors.

CONCLUSIONS

The Educational Scholars Program's implicit curriculum facilitated faculty-participants taking on mentor identity via opportunities it afforded to watch mentors, experiment with mentoring, and evaluate self as mentor, key ingredients for identity construction. Leaders of professional development programs can develop faculty as mentors by capitalizing on what faculty-participants learn in the implicit curriculum and deliberately structuring postgraduation mentoring opportunities.

A Structured Framework and Resources to Use to Get Your Medical Education Work Published

INTRODUCTION

Medical educators often have great ideas for medical education scholarship but have difficulty converting their educational abstract or project into a published manuscript.

METHODS

During this workshop, participants addressed common challenges in developing an educational manuscript. In small-group case scenarios, participants discovered the importance of the "So what?" in making the case for their project. Incorporating conceptual frameworks, participants chose appropriate outcome metrics, discussed how to frame the discussion section, and ensured appropriate journal fit. After each small-group exercise, large-group discussions allowed the small groups to report back so that facilitators could highlight and reinforce key learning points. At the conclusion of the workshop, participants left with a checklist for creating an educational manuscript and an additional resources document to assist them in avoiding common pitfalls when turning their educational abstract/project into a publishable manuscript.

RESULTS

This workshop was presented in 2016 and 2017. Presenter evaluations were completed by 33 participants; 11 completed conference evaluations. The mean overall rating on presenter evaluations was 4.55 out of 5, while the conference evaluations mean was 3.73 out of 4. Comments provided on both evaluation tools highlighted the perceived effectiveness of the delivery and content. More than 50% of respondents stated that they planned to incorporate the use of conceptual frameworks in future work.

DISCUSSION

This workshop helped participants address common challenges by providing opportunities for hands-on practice as well as tips and resources for use when submitting a medical education manuscript for publication.

The Impact of a National Faculty Development Program Embedded Within an Academic Professional Organization

A sizeable literature describes the effectiveness of institution-based faculty development programs in nurturing faculty educators as scholars, but national programs are less common and seldom evaluated. To fill this role, the Educational Scholars Program (ESP) was created within the Academic Pediatric Association (APA) in 2006. It is a national, three-year, cohort-based certification program focused on fostering educational scholarship. This article describes the development and outcomes of an innovative program embedded within the framework of a national professional organization, and offers a model for potential adaptation by similar organizations to enhance their support of educators.After 10 years, 171 scholars have enrolled in the ESP, and 50 faculty have participated. Scholars are assigned a faculty advisor and participate in three full-day sessions at a national meeting; online, interactive learning modules; and a mentored, scholarly project. The program receives support from the APA in four organizational frames: structural, human resource, political, and symbolic. The self-perceived scholarly proficiency of the scholars in Cohort 1 increased significantly over time, and their productivity and collaborations increased during and after the program. Scholars wrote enthusiastically about their experience in yearly and postprogram evaluations. In interviews, eight past APA presidents explained that the ESP strengthened the APA's mission, created new leaders, and provided a new model for other APA programs. Outcomes of the ESP suggest that a longitudinal faculty development program embedded within a national professional organization can create a social enterprise not only within the organization but also within the broader national community of educator-scholars.

Building Sustainable Professional Development Programs: Applying Strategies From Implementation Science to Translate Evidence Into Practice

INTRODUCTION

Multisite and national professional development (PD) programs for educators are challenging to establish. Use of implementation science (IS) frameworks designed to convert evidence-based intervention methods into effective health care practice may help PD developers translate proven educational methods and models into successful, well-run programs. Implementation of the national Educational Scholars Program (ESP) is used to illustrate the value of the IS model.

METHODS

Four adaptable elements of IS are described: (1) replication of an evidence-based model, (2) systematic stages of implementation, (3) management of implementation using three implementation drivers, and (4) demonstration of program success through measures of fidelity to proven models and sustainability.

RESULTS

Implementation of the ESP was grounded on five established principles and methods for successful PD. The process was conducted in four IS stages over 10 years: Exploration, Installation, Initial Implementation, and Full Implementation. To ensure effective and efficient processes, attention to IS implementation drivers helped to manage organizational relationships, build competence in faculty and scholars, and address leadership challenges. We describe the ESP's fidelity to evidence-based structures and methods, and offer three examples of sustainability efforts that enabled achievement of targeted program outcomes, including academic productivity, strong networking, and career advancement of scholars.

DISCUSSION

Application of IS frameworks to program implementation may help other PD programs to translate evidence-based methods into interventions with enhanced impact. A PD program can follow systematic developmental stages and be operationalized by practical implementation drivers, thereby creating successful and sustainable interventions that promote the academic vitality of health professions educators.

Discussing the undiscussable with the powerful: why and how faculty must learn to counteract organizational silence

Dialogue is essential for transforming institutions into learning organizations, yet many well-known characteristics of academic health centers (AHCs) interfere with open discussion. Rigid hierarchies, intense competition for resources, and the power of peer review in advancement processes all hamper difficult conversations, thereby contributing to organizational silence, and at great cost to the institution. Information necessary for critical decisions is not shared, individuals and the organization do not learn from mistakes, and diverse perspectives from those with less power are not entertained, or worse, are suppressed. When leaders become more skilled at inviting multiple perspectives and faculty more adept at broaching difficult conversations with those in power, differences are more effectively addressed and conflicts resolved. In this article, the authors frame why this skill is an essential competency for faculty and leaders alike and provide the following recommendations to institutions for increasing capacity in this area: (1) develop leaders to counteract organizational silence, (2) develop faculty members' skills in raising difficult issues with those in positions of power, and (3) train mentors to coach others in raising difficult conversations. The vitality of AHCs requires that faculty and institutional leaders develop relational communication skills and partner in learning through challenging conversations.

Evaluating educators using a novel toolbox: applying rigorous criteria flexibly across institutions

Valuing faculty as educators is essential for medical schools to fulfill their unique mission of educating physicians. The 2006 Consensus Conference on Educational Scholarship, sponsored by the Association of American Medical Colleges (AAMC) Group on Educational Affairs, provided educators seeking academic promotion with a portfolio-based format for documenting activities in five domains, using evidence of quantity, quality, a scholarly approach, and educational scholarship. Yet, the lack of a rigorous, widely accepted system to assess educator portfolio submissions during the promotion and tenure process continues to impede the ability to fully value educators and educational scholars.The AAMC Task Force on Educator Evaluation was formed in 2010 to establish consensus guidelines for use by those responsible for the rigorous evaluation of the educational contributions of faculty. The task force delineated the educational contributions currently valued by institutions and then fulfilled its charge by creating the Toolbox for Evaluating Educators, a resource which contains explicit evidence-based criteria to evaluate faculty in each of the five domains of educator activity. Adoption of such criteria is now the rate-limiting step in using a fair process to recognize educators through academic promotion. To inform institutional review and implementation of these criteria, this article describes the iterative, evidence- and stakeholder-based process to establish the criteria. The authors advocate institutional adoption of these criteria so that faculty seeking academic promotion as educators, like their researcher colleagues, can be judged and valued using established standards for the assessment of their work.

The value of speed mentoring in a pediatric academic organization

OBJECTIVE

A reliable and supportive mentor is indispensable to the career development of successful academic professionals. The Academic Pediatric Association (APA) utilized a speed mentoring format at the 2012 Pediatric Academic Societies meeting to enhance mentoring potential. We sought to evaluate the structure of the speed mentoring event and to determine the benefits and impact from the perspectives of the mentors and mentees.

METHODS

Sixty mentees were matched with 60 mentors within various tracks. Each mentee met with 6 mentors for 10 minutes for each dyad. Participants were then asked to complete a survey 1 to 4 weeks after the event. Survey items included expectation, impact, and value of the experience along with potential for ongoing mentoring relationships.

RESULTS

Fifty-four (90%) of the 60 mentees and 52 (87%) of 60 of the mentors completed the evaluation. Mentees stated that the event allowed them to receive advice from multiple mentors in a short time period. Mentors appreciated that they gained new insights, reflected on their own careers, and were able to give back to their field. Both mentees and mentors agreed that the time was well spent, would participate again, and identified chemistry as a major factor in pursuing an ongoing relationship.

CONCLUSIONS

This national speed mentoring event provided an innovative, fun, and time-efficient mechanism to establish connections, network, and determine whether chemistry existed for potential mentor-mentee relationships. Further study should evaluate whether it can be used in other venues and lead to the development of lasting mentor-mentee relationships.

Perspective: Toward a competency framework for faculty

Today, faculty in academic medicine face challenges in all three mission areas--research, education, and patient care--and require a broad set of competencies to survive in this changing environment. To support faculty and to design assessments that match new expectations, the authors argue that it is essential to capture the full scope of skills, knowledge, and behaviors necessary for a successful faculty member. Thus, it is timely to explore and define competencies for faculty in academic medicine. The authors describe three approaches to identifying faculty competencies. Each reveals diverse but overlapping sets of competency domains, reflecting the breadth of activities expected of today's faculty. To organize these competencies into a coherent framework, the authors propose a model based on a typology of competency. A key feature of the model is the division between occupational competencies, which are largely role-specific, and personal competencies, which are necessary for all faculty. A competency framework also must be developmental, to reflect the growth in skills, knowledge, and behaviors from trainee to expert and to allow for an individual's changing roles over a career. Such a competency framework will inform professional development activities and require assessment of competence. The generation of competencies also will reveal areas of faculty practice that are poorly measured, requiring new tools to be incorporated into existing processes of faculty evaluation. The authors provide general principles to guide the identification of a competency framework for faculty and invite the academic medicine community to engage in further discussion.

Twelve tips for conducting collaborative research in medical education

BACKGROUND

Developing and ensuring successful collaborative research in medical education is no small task, but the rewards to researchers can be great. Collaborative research in medical education offers significant opportunities for investigators who wish to pool limited resources and expand professional networks. Despite this, collaboration often occurs without advance planning for the logistical aspects of collaborative work.

AIMS AND METHODS

These 12 tips, derived from developing and presenting a session on strategies for effective collaboration conducted at a national meeting, will assist readers who are planning to collaborate or are already engaged in collaborative scholarship. The tips are organized into items to consider in three phases of collaboration: planning, implementation, and dissemination of outcomes.

RESULTS AND CONCLUSIONS

From selecting a topic to recognizing when the collaboration should end, the tips underscore the importance of good communication, and clear expectations for participants.

How mentoring relationships evolve: a longitudinal study of academic pediatricians in a physician educator faculty development program

INTRODUCTION

Mentoring is increasingly recognized as central to career development. Less attention has been paid, however, to how mentoring relationships evolve over time. To provide a more complete picture of these complex relationships, the authors explored mentoring from a mentee's perspective within the context of a three-year faculty development program in which the mentor provided specific expertise to assist the mentee in completing a scholarly educational project.

METHODS

Using an evolving focus group design, the authors interviewed mentee groups in 2007-2009 inclusive. Transcripts were coded inductively; codes were revised as data patterns became more apparent. Preliminary assertions about the answers to guiding questions were made; the trustworthiness of the assertions was assessed via member check.

RESULTS

Mentees offered a variety of reasons for choosing their project mentor, including proximity, familiarity, and mentor expertise. There was a dyadic relationship with the project mentor in year 1, a broader collaboration with multiple senior mentors in year 2, and mentoring among program peers in year 3. Mentees benefitted from mentors' supportive behaviors and, to a lesser extent, mentors' challenging behaviors.

CONCLUSION

Mentoring relationships, in the context of this faculty development program, tended not to be an exclusive dyadic connection but rather a constellation of relationships that evolved over time and included peer mentoring. The complex reality of these relationships challenges the application of traditional mentoring models and suggests unique considerations in developing mentoring programs designed to meet the needs of faculty in academic medicine.

The essential value of projects in faculty development

Projects--planned activities with specific goals and outcomes--have been used in faculty development programs to enhance participant learning and development. Projects have been employed most extensively in programs designed to develop faculty as educators. The authors review the literature and report the results of their 2008 study of the impact of projects within the Pennsylvania State University College of Medicine Junior Faculty Development Program, a comprehensive faculty development program. Using a mixed-methods approach, the products of project work, the academic productivity of program graduates, and the impact of projects on career development were analyzed. Faculty who achieved the most progress on their projects reported the highest number of academic products related to their project and the highest number of overall academic achievements. Faculty perceived that their project had three major effects on their professional development: production of a tangible outcome, development of a career focus, and development of relationships with mentors and peers. On the basis of these findings and a review of the literature, the authors conclude that projects are an essential element of a faculty development program. Projects provide a foundation for future academic success by enabling junior faculty to develop and hone knowledge and skills, identify a career focus and gain recognition within their community, generate scholarship, allocate time to academic work, and establish supportive relationships and collaborative networks. A list of best practices to successfully incorporate projects within faculty development programs is provided.

Analgesic properties of oral sucrose during routine immunizations at 2 and 4 months of age

OBJECTIVE

The purpose of this work was to evaluate the analgesic properties of oral sucrose during routine immunizations in infants at 2 and 4 months of age.

PATIENTS AND METHODS

A prospective, randomized, placebo-controlled clinical trial was conducted at a pediatric ambulatory care clinic. One-hundred healthy term infants scheduled to receive routine immunizations were recruited, randomly stratified into 2- or 4-month study groups, and further randomly assigned to receive 24% oral sucrose and pacifier or the sterile water control solution. The study preparations were administered 2 minutes before the combined diphtheria-tetanus-acellular pertussis, inactivated polio vaccine, and hepatitis B vaccine. Haemophilus influenzae type b vaccine was administered 3 minutes after the combined injection, followed by the pneumococcal conjugate vaccine, 2 minutes after the H. influenzae type b injection. The University of Wisconsin Children's Hospital Pain Scale measured serial acute pain responses for the treatment and control groups at baseline and 2, 5, 7, and 9 minutes after solution administration. Repeated-measures analysis of variance examined between-group differences and within-subject variability of treatment effect on overall pain scores.

RESULTS

Two- and 4-month-old infants receiving oral sucrose (n = 38) displayed reductions in pain scores 2 minutes after solution administration compared with 2- and 4-month-old infants in the placebo group (n = 45). Between-group comparisons for the oral sucrose and placebo groups showed lower pain responses at 5, 7, and 9 minutes after solution administration. The oral sucrose and placebo groups demonstrated their highest mean pain score at 7 minutes, with a mean pain score of 3.8 and 4.8, respectively. At 9 minutes, the placebo group had a mean pain score of 2.91 whereas the mean pain score for the oral sucrose group returned to near baseline, reflecting a 78.5% difference in mean pain score (oral sucrose - placebo) relative to the placebo mean.

CONCLUSIONS

Oral sucrose is an effective, easy-to-administer, short-acting analgesic for use during routine immunizations.

Functional mentoring: a practical approach with multilevel outcomes

INTRODUCTION

Mentoring is a central component of professional development. Evaluation of "successful" mentoring programs, however, has been limited and mainly focused on measures of satisfaction with the relationship. In today's environment, mentoring programs must produce tangible outcomes to demonstrate success. To address this issue, the authors advance the framework of functional mentoring combined with measurement of outcomes at multiple levels.

METHODS

The mentoring program is embedded within an intensive, continuing medical education (CME) accredited faculty development program. Survey methodology is used to collect qualitative and quantitative data at the start, midpoint, and end of the program and longitudinally. Participants in 4 years of the program were surveyed.

RESULTS

In 4 years, 165 faculty participated in the program. Respondents were highly satisfied with the pairings: 85% of junior faculty believed their mentor had a significant effect on their projects. Junior faculty reported a significant enhancement of skills related to initiating and negotiating a new mentoring relationship (85%) and stated that their project would have a significant impact on their career (92%) and on the department or institution (86%).

DISCUSSION

The success of this mentoring program is demonstrated at multiple levels. The key outcome of functional mentoring is the project. Projects are aligned with professional responsibilities and with institutional missions. The project contributes to the individual's dossier and adds value to the institution. Functional mentoring is a practical approach that allows measurable results at multiple levels.

Teaching beliefs of medical educators: perspectives on clinical teaching in pediatrics

BACKGROUND

Most who teach in clinical settings see themselves primarily as clinicians or physicians responsible for patient care and only secondarily as educators. The education literature suggests that teaching predominantly operates at a tacit level, where teachers rely on core beliefs to guide their practice, and actually spend little time in reflective practice. Given the lack of research on how medical educators in clinical settings view their teaching, the purpose of this qualitative study was to explore the teaching beliefs of faculty in a pediatrics department in a college of medicine.

METHODS

Using a Teaching Perspectives Inventory, observations and in-depth interviews, a complex picture was revealed about teaching beliefs of medical educators.

RESULTS

Due to contextual constraints of the clinical setting (e.g., time, competing stakeholders) that requires primary attention to patient care, they describe 'teaching on the fly'. There is a strong emphasis on: delivering content; encouraging thinking among students; providing questioning and engaging learning experiences; and respecting students as learners.

CONCLUSIONS

The implications of these beliefs are significant and indicate that faculty can benefit from opportunities that make their beliefs about teaching more conscious, particularly in determining how best to prepare future physicians to teach in clinical settings.

Improving influenza vaccination rates for children through year-round scheduling

OBJECTIVE

Barriers to influenza vaccination negatively impact immunization rates contributing to the morbidity and mortality from influenza. This study sought to determine if 1) the availability of year-round scheduling of annual autumn/winter influenza vaccination was associated with improved immunization rates for 2 high-risk populations of children and 2) this system was associated with early season vaccine administration.

METHODS

A retrospective cohort analysis was utilized to compare immunization rates during the 2003-2004 and 2004-2005 seasons. Billing records were used to determine eligible patients and vaccine receipt. A single, pediatric practice was studied, and two groups of patients were analyzed: 1) infants aged 6-23 months and 2) children < 21 years old with asthma. As opposed to Year 1, in Year 2 appointments in "flu clinics" for the following autumn became available 7 months prior to when vaccine became available. Patients and providers could therefore schedule an immunization appointment throughout the year.

RESULTS

In Year 1, 552/1365 (40.4%) infants received at least 1 dose of influenza vaccine compared to 940/1265 (74.3%) in Year 2 (p < 0.001). For patients with asthma, 309/1332 (23.2%) received at least 1 dose of vaccine in Year 1 compared with 522/1489 (35.1%) in Year 2 (p <0.001). Both groups also achieved higher immunization rates between September and November in Year 2 (p < 0.001).

CONCLUSIONS

Year-round scheduling of influenza vaccination was associated with improved immunization rates in two high-risk populations, and may remove the barrier of scheduling difficulty, allow for a consistent year-round message from providers, and improve timing of vaccine administration.

Rural and urban children with asthma: are school health services meeting their needs?

OBJECTIVE

Children with asthma spend a large portion of their day in school, and the extent to which public schools are prepared to meet their health needs is an important issue. The objective of this study was to identify asthma policies and practices in rural and urban school settings and to compare them with current National Heart, Lung, and Blood Institute recommendations.

METHODS

A stratified random sample of school nurses who represented each of the 500 active Pennsylvania school districts were surveyed in 2004 concerning nurse staffing patterns, availability of asthma monitoring and treatment-related equipment, emergency preparedness, availability of asthma-related support and case management services, school-specific procedures including identification of children with asthma and accessibility of inhaler medication during school hours, presence and content of written asthma management plans, and perceived obstacles to asthma management in the school setting. Sampling weights were incorporated into the analyses to take the survey design into account.

RESULTS

The overall response rate was 76%, with a total of 757 surveys analyzed. In more than half of secondary schools and three quarters of elementary schools, nurses were present < 40 hours per week. Nearly 1 in 5 schools reported that staff who know what to do for a severe asthma attack were not always available. In 72% of rural schools, children were allowed to self-carry rescue inhalers, as compared with 47% of urban schools. Asthma management plans were on file for only 1 quarter of children with asthma, and important information often was omitted. Approximately half of the schools were equipped with peak flow meters and nebulizers, and spacers were available in 1 third of schools.

CONCLUSIONS

Improvements are needed to bring schools into compliance with current recommendations, including more consistent availability of knowledgeable staff, improved access to asthma monitoring and treatment-related equipment, more universal use of asthma management plans, and greater access to inhalers while at school, including increasing the proportion of children who are allowed to carry and self-administer inhaler medication.

Empowering junior faculty: Penn State's faculty development and mentoring program

Empowerment of faculty is essential for academic success. The Junior Faculty Development Program (JFDP), sponsored by the Office of Professional Development of the Penn State College of Medicine, was established in 2003 with the goal of promoting the development and advancement of junior faculty so they can achieve success in their academic careers. The program consists of two components: a curriculum in research, education, clinical practice, and career development, and an individual project completed under the guidance of a senior faculty mentor. The curriculum provides faculty with knowledge, skills, and resources. Mentoring provides relationships and support. Together, these elements combine to empower junior faculty to better manage their careers. The effectiveness of the program has been demonstrated by several measures: participants evaluated the program highly, demonstrated increases in their perceptions of their own abilities, and completed tasks important to the advancement of their careers. Participants stated they were better prepared to advance their academic careers and that the individual projects would contribute to their career advancement. On the basis of this experience, the authors suggest that faculty development programs should empower faculty so that they can more effectively chart a successful career in academic medicine. This report describes an empowerment model, and the design, implementation, and evaluation of the Junior Faculty Development Program in 2003-04 and 2004-05. The authors offer this program as a model for the benefit of other institutions and for one of their most valuable assets: junior faculty.