Curated Collections for Educators: Five Key Papers about Program Evaluation

Beyond the CLAIM: A comprehensive needs assessment strategy for creating an Advanced Medical Education Research Training Program (ARMED-MedEd)

BACKGROUND

The health professions education (HPE) landscape has shifted substantively with increasing professionalization of research and scholarship. Clinician educators seeking to become competitive in this domain often pursue fellowships and master's degrees in HPE, but there are few resources for the continuing professional development (CPD) of those who seek to continue developing their scholarly practice within HPE. Acknowledging the multiple players in this landscape, the authors sought to design a new "beyond beginners" HPE research program using a novel needs assessment planning process.

METHODS

The authors developed and conducted a new three-phase, five-step process that sets forth a programmatic approach to conducting a needs assessment for a CPD course in HPE research. The five steps of the CLAIM method are: Competitive analysis, Literature review with thematic analysis, Ask stakeholders, Internal review by experts, and Mapping of a curriculum. These steps are organized into three phases (Discovery, Convergence, and Synthesis).

RESULTS

Over a 12-month period, the authors completed a comprehensive needs assessment. The CLAIM process revealed that longitudinal digital connection, diverse and in depth exposure to HPE research methods, skills around scholarly publishing, and leadership and management of research would be beneficial to our design.

CONCLUSIONS

The CLAIM method provided scaffolding to help the authors create a robust curriculum that adopts a scholarly approach for developing a HPE research course. This needs assessment methodology may be useful in other CPD contexts.

Curated collections for educators: Six key papers on teaching procedural skills

BACKGROUND

Competence in teaching procedural skills is required for faculty in all specialties. Regardless of involvement in undergraduate medical education (UME) versus graduate medical education (GME), faculty will likely be involved in teaching procedures to novice learners at some point, with the goal of having the learner achieve graduated independence and technical competence in a skill set. A large body of literature exists addressing the best practices for teaching and maintaining procedural skills. We searched for articles that describe the best practices for teaching procedural skills to all levels of learners.

METHODS

We conducted a literature search for papers on procedural skills training and teaching. We also made a call for papers on social media from members of the online #MedEd and #FOAMed communities. Once a list of the articles was compiled, we conducted a three-round modified Delphi process to identify those illustrating best practices for teaching procedural skills by both junior and senior faculty.

RESULTS

We identified 98 relevant articles on the topic of procedural skills training. Six articles were deemed to be highly relevant after three rounds of the modified Delphi. Best practices included using an established educational framework when designing procedural skills teaching sessions, providing positive feedback to learners with opportunities for improvement, and demonstrating the procedure to the learners.

CONCLUSIONS

Medical educators should employ evidence-based practices when designing and delivering procedural skills sessions. Educational frameworks provide faculty developers and facilitators with an organized approach to teaching these sessions. Maintenance of procedural skills over time is key; faculty can utilize simulation-based procedural training and deliberate practice to prevent decay of learned skills.

Curated Collections for Educators: Five Key Papers on Clinical Teaching

The ability to teach in the clinical setting is of paramount importance. Clinical teaching is at the heart of medical education, irrespective of the learner’s level of training. Learners desire and need effective, competent, and thoughtful clinical teaching from their instructors. However, many clinician-educators lack formal training on this important skill and thus may provide a variable experience to their learners. Although formal training of clinician-educators is standard and required in many other countries, the United States has yet to follow suit, leaving many faculty members to fend for themselves to learn these important skills. 

In September 2018, the Academic Life in Emergency Medicine (ALiEM) 2018-2019 Faculty Incubator program discussed the topic of clinical teaching techniques. We gathered the titles of papers that were cited, shared, and recommended within our online discussion forum and compiled the articles pertaining to the topic of clinical teaching techniques. To augment the list, the authors did a formal literature search using the search terms “teaching techniques", "clinical teaching", "medical education", "medical students", and "residents” on Google Scholar and PubMed. Finally, we posted a call for important papers on the topic of clinical teaching techniques on Twitter.

Through this process, we identified 48 core articles on the topic of clinical teaching. We conducted a modified Delphi methodology to identify the key papers on the topic. In this paper, we present the five highest-rated articles based on the relevance to junior faculty and faculty developers. This article will review and summarize the articles we found to be the most impactful to improve one’s clinical teaching skills. 

Curated Collections for Educators: Five Key Papers about Receiving Feedback in Medical Education

Introduction Feedback is a complex, multi-component interaction that is essential for academic development and advancement. Successful feedback requires active involvement from both the giver and receiver. However, research and guidance on the subject mostly center on the role of the provider of feedback. But the receiver of feedback holds the true power in this interaction, choosing how to interpret the information and deciding whether or not to incorporate the feedback to instill behavioral change. In this article, the authors aim to summarize five key papers related to receiving feedback, in order to outline both relevant information for emerging clinician-educators and discern ways to use this information for faculty development. Methods In order to generate a list of key papers that describe the importance of receiving feedback, the authors conducted a consensus-building process informed by social media sources. Key articles on receiving feedback were aggregated through a literature search. This list was further augmented via an open call on Twitter for important papers regarding receiving feedback. Through these processes, a list of 43 papers was created on the topic of receiving feedback in medical education. After compiling this preliminary list, the authorship group engaged in a modified Delphi approach to build consensus on selecting papers that best described the process of receiving feedback. Results We present the group's five most highly rated papers on the topic of receiving feedback in medical education. These papers were deemed essential and have also been summarized based on their relevance to junior faculty members and faculty developers. Conclusion While giving and receiving feedback are both vital for growth and development, much of the research focuses solely on giving feedback. However, receiving feedback is equally, if not more, important for instilling change in the learner. We explore the power of receiving feedback in medical education through five key papers that analyze the subject. We believe these papers can serve as great learning resources for both junior faculty members and faculty developers. They can assist the junior faculty to cultivate the ability to receive feedback and also serve as resources to aid senior faculty in building faculty-development sessions.

Faculty development program evaluation: a need to embrace complexity

Faculty development is essential for renewing and assisting faculty to maintain teaching effectiveness and adapt to innovations in Health Professions educational institutions. The evaluation of faculty development programs appears to be a significant step in maintaining its relevance and efficiency. Yet, little has been published on the specific case of faculty development program evaluation in spite of the availability of general program evaluation models. These models do not measure or capture the information educators want to know about outcomes and impacts of faculty development. We posit that two reasons account for this. The first is the evolving nature of faculty development programs as they adapt to current reforms and innovations. The second involves the limitations imposed by program evaluation models that fail to take into account the multiple and unpredictable outcomes and impacts of faculty development. It is generally accepted that the outcomes and impacts are situated at various levels, ranging from the individual to the institutional and cultural levels. This calls for evaluation models that better capture the complexity of the impacts of faculty development, in particular the reciprocal relationships between program components and outcomes. We suggest conceptual avenues, based on Structuration Theory, that could lead to identifying the multilevel impacts of faculty development.

Curated Collections for Clinician Educators: Five Key Papers on Graduated Responsibility in Residency Education

Introduction

The Accreditation Council for Graduate Medical Education calls graduated responsibility “one of the core tenets of American graduate medical education.” However, there is no clear set of resources for programs to implement a system of progressively increasing responsibilities for trainees. This project aimed to identify a set of high-yield papers on graduated responsibility for junior faculty members.

Methods

A study group of Academic Life in Emergency Medicine Faculty Incubator participants identified relevant literature on graduated responsibility via a comprehensive literature search and a call to the online medical education community; 59 total papers were identified. The most relevant and applicable were selected by the study group via a three-round modified Delphi process.

Results

Five key articles for junior faculty interested in implementing more robust graduated responsibility at their residency training program were selected and described here. Summaries of key points, along with considerations for faculty developers and relevance to junior faculty, are presented for each article.

Conclusions

The articles presented here provide a solid theoretical and practical basis for junior faculty to explore graduated responsibility. The five articles presented here provide the junior faculty with a toolkit to examine and improve their systems for assigning responsibilities in a graded fashion at their own institutions.

Curated Collections for Educators: Eight Key Papers about Feedback in Medical Education

Feedback is an essential part of learning, growth, and academic success. Junior faculty members are often unfamiliar with the grounding literature that defines feedback. Many times they receive little education on providing and receiving feedback, resulting in unhelpful "feedback" for both learners and program leadership alike. This article aims to summarize eight key papers on feedback, to outline relevant information for emerging clinician educators, and identify ways to use these resources for the faculty development.  In order to generate a list of key papers that describes the importance and significance of feedback, the authors conducted a consensus-building process to identify the top papers. In August and September, 2018, the 2018-2019 Academic Life in Emergency Medicine (ALiEM) Faculty Incubator program discussed the topic of feedback in medical education. A number of papers on the topic was highlighted. This list of papers was further augmented using the suggestions and expertise of guest experts who are leaders in the field of medical education and feedback. The authors also used social media to conduct an open call on Twitter for important papers regarding feedback (utilizing #meded, #Feedback hashtags). Via this process, a list of 88 key papers was identified on the topic of feedback in medical education. After compiling these papers, the authorship group engaged in a modified Delphi approach to build consensus on the top eight papers on feedback. These papers were deemed essential by the authors and have been summarized with respect to their relevance to junior faculty members and to faculty developers. In this manuscript, we present eight key papers addressing feedback in medical education with discussions and applications for junior faculty members and faculty developers. This list of articles that can serve to help junior clinician educators grow in their ability to give effective feedback and also serve as resources upon which senior faculty can design the faculty development sessions.

Comparative Analysis of Junior and Senior Clinician Educator Evaluation of Relevant Articles Within Medical Education

Introduction

It may be difficult for junior clinician educators (JCEs) to get a grasp of pertinent literature and determine which are most relevant to their learning, due to limited experience and lack of formalized system to rank all available resources with respect to their value for JCEs. Our study aimed to identify whether senior clinician educators (SCEs) and JCEs differ in their selection of what they perceive as key medical education articles.

Methods

As a part of the Academic Life in Emergency Medicine (ALiEM) Faculty Incubator program, we developed a series of primer articles for JCEs by identifying and discussing key articles within specific medical education arenas, which were designed to enhance the reader's educational growth. Each set of articles within the primer series were selected based on data collected from JCEs and SCEs, who ranked the specific articles with respect to their perceived relevancy to the JCEs. ANOVA analysis was performed for each of the series to determine whether there was a statistically significant difference between JCE and SCE rating of articles.

Results

Two-hundred-and-sixteen total articles were evaluated within the nine primer topics. No statistically significant difference was found between the rankings of papers by JCEs and SCEs (effect size: 0.06; 95% CI: -0.27 to 0.40). However, a subgroup analysis of the data found that three of the nine primers showed statistically significant divergence based on seniority (p < 0.05).

Conclusions

Based on the data, the involvement of JCEs in the consensus-building process was important in identifying divergence in views between JCEs and SCEs in one-third of cases. Our findings suggest that it is important to involve JCEs in selecting articles that are worthwhile for their learning, since SCEs may not fully understand their needs.

Curated Collections for Educators: Five Key Papers about Residents as Teachers Curriculum Development

The Accreditation Council for Graduate Medical Education (ACGME) requires residency programs to prepare residents to teach and assess medical students and other learners. In order to achieve this, many programs develop formal residents as teachers (RAT) curricula. Medical educators may seek the guidance of previously published literature during the development of RAT programs at their institutions. The authors sought to identify key articles published on the subject of RAT programs over the last 10 years. The authors utilized a formal literature search with the help of a medical librarian and identified additional articles from virtual discussions among the author group and an open call for articles on Twitter using the hashtag #MedEd. Virtual discussions occurred within an online community of practice, the Academic Life in Emergency Medicine (ALiEM) Faculty Incubator. The lead author conducted a four-round modified Delphi process among the author group in order to narrow the broad article list to five key articles on RAT programs. The authors summarize each article and provide considerations for junior faculty as well as faculty developers. Curriculum development and program evaluation should utilize established frameworks and evidence-based approaches. The papers identified by this Delphi process will help faculty use best practices when creating or revising new RAT curriculum. In addition, faculty tasked with guiding junior faculty in this process or creating faculty development programs around curriculum development will find these articles to be a great resource for building content.

Curated Collections for Educators: Five Key Papers on Evaluating Digital Scholarship

Traditionally, scholarship that was recognized for promotion and tenure consisted of clinical research, bench research, and grant funding. Recent trends have allowed for differing approaches to scholarship, including digital publication. As increasing numbers of trainees and faculty turn to online educational resources, it is imperative to critically evaluate these resources. This article summarizes five key papers that address the appraisal of digital scholarship and describes their relevance to junior clinician educators and faculty developers. In May 2017, the Academic Life in Emergency Medicine Faculty Incubator program focused on the topic of digital scholarship, providing and discussing papers relevant to the topic. We augmented this list of papers with further suggestions by guest experts and by an open call via Twitter for other important papers. Through this process, we created a list of 38 papers in total on the topic of evaluating digital scholarship. In order to determine which of these papers best describe how to evaluate digital scholarship, the authorship group assessed the papers using a modified Delphi approach to build consensus. In this paper we present the five most highly rated papers from our process about evaluating digital scholarship. We summarize each paper and discuss its specific relevance to junior faculty members and to faculty developers. These papers provide a framework for assessing the quality of digital scholarship, so that junior faculty can recommend high-quality educational resources to their trainees. These papers help guide educators on how to produce high quality digital scholarship and maximize recognition and credit in respect to receiving promotion and tenure.

Curated Collection for Educators: Five Key Papers about the Flipped Classroom Methodology

The flipped classroom (FC) pedagogy is becoming increasingly popular in medical education due to its appeal to the millennial learner and potential benefits in knowledge acquisition. Despite its popularity and effectiveness, the FC educational method is not without challenges. In this article, we identify and summarize several key papers relevant to medical educators interested in exploring the FC teaching methodology.

The authors identified an extensive list of papers relevant to FC pedagogy via online discussions within the Academic Life in Emergency Medicine (ALiEM) Faculty Incubator. This list was augmented by an open call on Twitter (utilizing the #meded, #FOAMed, and #flippedclassroom hashtags) yielding a list of 33 papers. We then conducted a three-round modified Delphi process within the authorship group, which included both junior and senior clinician educators, to identify the most impactful papers for educators interested in FC pedagogy.

The three-round modified Delphi process ranked all of the selected papers and selected the five most highly-rated papers for inclusion. The authorship group reviewed and summarized these papers with specific consideration given to their value to junior faculty educators and faculty developers interested in the flipped classroom approach.

The list of papers featured in this article serves as a key reading list for junior clinician educators and faculty developers interested in the flipped classroom technique. The associated commentaries contextualize the importance of these papers for medical educators aiming to optimize their understanding and implementation of the flipped classroom methodology in their teaching and through faculty development.