"I'll never play professional football" and other fallacies of self-assessment

Physician educators' perceptions of experiences contributing to teaching

INTRODUCTION

Physician educators are essential in training the next generation of physicians. However, physician educators' perspectives about what experiences they find beneficial to their teaching and the prevalence of these experiences remain unknown. Guided by social cognitive career theory (SCCT) and communities of practice (CoP), we explored what experiences physician educators perceive as beneficial in preparing them to teach.

METHODS

In 2019, the Uniformed Services University School of Medicine in the United States surveyed its physician alumni to understand their education experiences during medical school, their current career path and what has contributed to their teaching role. Content analysis was applied to extract themes across the text response. Chi-square analysis was applied to examine if perceived contributing factors vary based on physician educators' gender, specialty and academic ranks.

RESULTS

The five most prevalent contributing factors participants (n = 781) identified are (1) experiences gained during residency and fellowship (29.8%), (2) teaching as faculty member (28.9%) and (3) class experiences and peer interaction during medical school (26%). We organised three themes that reflected major avenues of how physician educators acquire teaching skills: reflection about quality teaching, journey as learners and learning by doing. Gender and clinical specialty were differentially associated with contributing factors such as faculty development and meta-reflection.

CONCLUSION

The results are in line with theories of SCCT and CoP, in which we identified self-directed learning and regulation in shaping physician educators' teaching. The findings also revealed gaps and potential contexts for more formalised teaching practices to develop physician educators.

Knowledge Mapping and Global Trends in the Field of the Objective Structured Clinical Examination: Bibliometric and Visual Analysis (2004-2023)

BACKGROUND

The Objective Structured Clinical Examination (OSCE) is a pivotal tool for assessing health care professionals and plays an integral role in medical education.

OBJECTIVE

This study aims to map the bibliometric landscape of OSCE research, highlighting trends and key influencers.

METHODS

A comprehensive literature search was conducted for materials related to OSCE from January 2004 to December 2023, using the Web of Science Core Collection database. Bibliometric analysis and visualization were performed with VOSviewer and CiteSpace software tools.

RESULTS

Our analysis indicates a consistent increase in OSCE-related publications over the study period, with a notable surge after 2019, culminating in a peak of activity in 2021. The United States emerged as a significant contributor, responsible for 30.86% (1626/5268) of total publications and amassing 44,051 citations. Coauthorship network analysis highlighted robust collaborations, particularly between the United States and the United Kingdom. Leading journals in this domain-BMC Medical Education, Medical Education, Academic Medicine, and Medical Teacher-featured the highest volume of papers, while The Lancet garnered substantial citations, reflecting its high impact factor (to be verified for accuracy). Prominent authors in the field include Sondra Zabar, Debra Pugh, Timothy J Wood, and Susan Humphrey-Murto, with Ronaldo M Harden, Brian D Hodges, and George E Miller being the most cited. The analysis of key research terms revealed a focus on "education," "performance," "competence," and "skills," indicating these are central themes in OSCE research.

CONCLUSIONS

The study underscores a dynamic expansion in OSCE research and international collaboration, spotlighting influential countries, institutions, authors, and journals. These elements are instrumental in steering the evolution of medical education assessment practices and suggest a trajectory for future research endeavors. Future work should consider the implications of these findings for medical education and the potential areas for further investigation, particularly in underrepresented regions or emerging competencies in health care training.

More Than Maintaining Competence: A Qualitative Study of How Physicians Conceptualize and Engage in Lifelong Learning

Purpose

Physicians have a professional responsibility to engage in lifelong learning. Some of this lifelong learning is required to maintain licensure and certification. Yet, this conceptualization captures only a small portion of the content areas and learning processes that physicians need to engage with to ensure quality patient care. Additionally, purposes beyond regulatory requirements and professional obligations likely drive physicians lifelong learning, though these purposes have not been explored. Given the centrality of lifelong learning to quality patient care, our study explores how physicians conceptualize and engage in lifelong learning.

Method

We conducted a qualitative interview study using an interpretivist approach. In 2019, we recruited 34 academic physicians from one institution. We analyzed our data to identify themes related to conceptualization of purposes, content areas, and processes of lifelong learning and actual lifelong learning practices.

Results

We interpreted participants' descriptions and examples of lifelong learning as serving three purposes: maintaining competence, supporting personal growth and fulfillment, and engaging in professional stewardship. Much of participants' discussion of lifelong learning centered around keeping up to date with medical knowledge and clinical/procedural skills, though some also mentioned efforts to improve communication, leadership, and teamwork. Participants engaged in lifelong learning through contextual, social, and individual processes.

Discussion

Academic physicians engage in lifelong learning for reasons beyond maintaining competence. Medical knowledge and clinical/procedural skills receive most attention, though other areas are recognized as important. Our findings highlight opportunities for a broader, more comprehensive approach to lifelong learning that spans all areas of medical practice.

The dynamics of self-monitoring in medicine: Safety, efficiency and clinical implications

.@MedEdDoc et al. delve into how the concepts of 'safety' and 'efficiency' in self‐monitoring can be used to influence clinical practice and #MedEd.

E-portfolio as an effective tool for improvement of practitioner nurses' clinical competence

BACKGROUND

Clinical competence is essential for providing effective patient care. Clinical Governance (CG) is a framework for learning and assessing clinical competence. A portfolio is a work-placed-based tool for monitoring and reflecting on clinical practice. This study aimed to investigate the effect of using an e-portfolio on the practitioner nurses' competence improvement through the CG framework.

METHODS

This was a quasi-experimental study with 30 nurses in each intervention and control group. After taking the pretests of knowledge and performance, the participants attended the in-person classes and received the educational materials around CG standards for four weeks. In addition, nurses in the intervention group received the links to their e-portfolios individually and filled them out. They reflected on their clinical practice and received feedback. Finally, nurses in both groups were taken the post-tests.

RESULTS

Comparing the pre-and post-test scores in each group indicated a significant increase in knowledge and performance scores. The post-test scores for knowledge and performance were significantly higher in the intervention group than in the control one, except for the initial patient assessment.

CONCLUSION

This study showed that the e-portfolio is an effective tool for the improvement of the nurses' awareness and performance in CG standards. Since the CG standards are closely related to clinical competencies, it is concluded that using portfolios effectively improves clinical competence in practitioner nurses.

Virtual Patient Simulation Offers an Objective Assessment of CME Activity by Improving Clinical Knowledge and the Levels of Competency of Healthcare Providers

The main goal of continuing medical education (CME) is to help healthcare providers (HCP) improve their knowledge and levels of competency with an ultimate enhancement of their performance in practice. Despite the long and well-intentional history of CME, the proof of success (based on improved clinical outcomes) is difficult to obtain objectively. In the past several years, the traditional CME world has been disrupted by replacing multiple-choice questions with virtual simulation. We utilised an innovative, next-generation virtual patient simulation (VPS) platform to develop objective measures to assess the success of educational activities that can be applied to the CME. This VPS platform was used at five distinct educational events designed to assess learners' knowledge and competency in the guideline-driven management of Type 2 diabetes, hyperlipidaemia, and hypertension. A total of 432 learners (medical doctors, nurse practitioners, and clinical pharmacists) participated in these educational events of whom 149 went through two consecutive cases with a similar clinical picture and educational goals. Their ability to achieve glycaemic, lipid, and blood pressure control improved significantly as they moved from the first to the second case. The participants improved their test performance in all categories - between 5 and 38%, achieving statistically significant increases in the many goals examined. In conclusion, this study employed the pioneering application of technology to produce, collect and analyse the VPS data to evaluate objectively educational activities. This VPS platform allows not only an objective assessment of the effectiveness of the CME activity but also provides timely and helpful feedback to both learners and providers of a given educational event.

Finding our Voice: Evaluation of Goal Setting Using the Habits of Preceptors Rubric in Terms of Focus, "SMARTness," and Impact

INTRODUCTION

Effective leadership propels teams from effectiveness to greatness and is accomplished when everyone achieves and contributes their full potential, or "voice." The Clinician Educators Program Teaching and Learning Curriculum fosters preceptor development using the Habits of Preceptors Rubric (HOP-R) to guide participants in finding their precepting "voice." After the HOP-R self-assessment, participants select a habit of focus (HOF) and craft a SMART (specific/measurable/achievable/relevant/time-bound) goal. This report describes a pilot rubric, SMART-EP (emotional intelligence(EI)/professionalism), exploring goal "SMARTness" alongside change (impact) in participants' perceived precepting capabilities.

METHODS

HOP-R self-ratings (2018-2020) and HOF/SMART goals (2019-2020) were retrospectively reviewed by two raters. Perceived preceptor capabilities were measured by analyzing the change in self-assessed habit level ratings between the first/fourth-quarter surveys. SMART goals were categorized by HOF and inclusion of SMART-EP components. Participants were guided in the inclusion of SMART, but not -EP, components.

RESULTS

In aggregate, 120 participants completed HOP-R surveys (2018-2020). Within-subject changes across all 11 habits were significant ( P < .001). For the SMART-EP rubric analysis (2019-2020), 71 participants had an average "SMARTness" score of 3.92 (of 5) with corresponding interrater reliability of 0.91. Goals included 2.77 (of 4) EI traits and 1.72 (of 3) professionalism components.

DISCUSSION

The SMART-EP rubric provided insights into preceptor development opportunities among participants. Beyond SMART components, participants often included elements of EI and professionalism. Ratings confirm and support the consistency of the HOP-R as a tool to assess precepting habits.

Feedback that Lands: Exploring How Residents Receive and Judge Feedback During Entrustable Professional Activities

Introduction

Receiving feedback from different types of assessors (e.g., senior residents, staff supervisors) may impact trainees' perceptions of the quantity and quality of data during entrustable professional activity (EPA) assessments. We evaluated the quality of EPA feedback provided by different assessors (senior residents, chief medical residents/subspecialty residents, and staff) and explored residents' judgements of the value of this feedback.

Methods

From a database of 2228 EPAs, we calculated the frequency of contribution from three assessor groups. We appraised the quality of 60 procedure-related EPAs completed between July 2019 and March 2020 using a modified Completed Clinical Evaluation Report Rating (CCERR) tool. Next, we asked 15 internal medicine residents to sort randomly selected EPAs according to their judgements of value, as an elicitation exercise before a semi-structured interview. Interviews explored participants' perceptions of quality of written feedback and helpful assessors.

Results

Residents completed over 60% of EPA assessments. We found no difference in modified-CCERR scores between the three groups. When judging EPA feedback value, residents described a process of weighted deliberation, considering perceived assessor characteristics (e.g., credibility, experience with EPA system), actionable written comments, and their own self-assessment.

Discussion

Like other recent studies, we found that residents contributed most to procedure-related EPA assessments. To the established list of factors influencing residents' judgements of feedback value, we add assessors' adherence to, and their shared experiences of being assessed within, EPA assessment systems. We focus on the implications for how assessors and leaders can build credibility in themselves and in the practices of EPA assessments.

Nurturing professional behaviours and ethical practice: From students to professionals

Medical professionalism refers to the set of values, behaviours, and ethical principles that guide the conduct of medical professionals in their interactions with patients, peers and the broader healthcare system. From a training perspective, it is easier to focus on striving to achieve excellence in medical practice and meeting “industry” accepted standards than to remediate unprofessional behaviours of practitioners.1 Medical professionalism encompasses a commitment to providing high-quality patient care, maintaining ethical standards, and upholding the trust and respect of patients and the community. This is essential for ensuring patient safety, promoting effective healthcare delivery, and preserving the integrity of the medical profession. Systematic reviews published on unprofessional behaviours of medical students, such as failure to engage, dishonest and disrespectful behaviours, and lack of self-awareness have shown to have a negative impact on peers, teachers and patients leading to poor teamwork and provision of quality care.2

A scoping review of self-monitoring in graduate medical education

BACKGROUND

Physicians and physicians-in-training have repeatedly demonstrated poor accuracy of global self-assessments, which are assessments removed from the context of a specific task, regardless of any intervention. Self-monitoring, an in-the-moment self-awareness of one's performance, offers a promising alternative to global self-assessment. The purpose of this scoping review is to better understand the state of self-monitoring in graduate medical education.

METHODS

We performed a scoping review following Arksey and O'Malley's six steps: identifying a research question, identifying relevant studies, selecting included studies, charting the data, collating and summarising the results and consulting experts. Our search queried Ovid Medline, Web of Science, PsychINFO, Eric and EMBASE databases from 1 January 1999 to 12 October 2022.

RESULTS

The literature search yielded 5363 unique articles. The authors identified 77 articles for inclusion. The search process helped create a framework to identify self-monitoring based on time and context dependence. More than 20 different terms were used to describe self-monitoring, and only 13 studies (17%) provided a definition for the equivalent term. Most research focused on post-performance self-judgements of a procedural skill (n = 31, 42%). Regardless of task, studies focused on self-judgement (n = 66, 86%) and measured the accuracy or impact on performance of self-monitoring (n = 41, 71%). Most self-monitoring was conducted post-task (n = 65, 84%).

CONCLUSION

Self-monitoring is a time- and context-dependent phenomenon that seems promising as a research focus to improve clinical performance of trainees in graduate medical education and beyond. The landscape of current literature on self-monitoring is sparse and heterogeneous, suffering from a lack of theoretical underpinning, inconsistent terminology and insufficiently clear definitions.

Assessment Practices in Continuing Professional Development Activities in Health Professions: A Scoping Review

INTRODUCTION

In continuing professional development (CPD), educators face the need to develop and implement innovative assessment strategies to adhere to accreditation standards and support lifelong learning. However, little is known about the development and validation of these assessment practices. We aimed to document the breadth and depth of what is known about the development and implementation of assessment practices within CPD activities.

METHODS

We conducted a scoping review using the framework proposed by Arksey and O'Malley (2005) and updated in 2020. We examined five databases and identified 1733 abstracts. Two team members screened titles and abstracts for inclusion/exclusion. After data extraction, we conducted a descriptive analysis of quantitative data and a thematic analysis of qualitative data.

RESULTS

A total of 130 studies were retained for the full review. Most reported assessments are written assessments (n = 100), such as multiple-choice items (n = 79). In 99 studies, authors developed an assessment for research purpose rather than for the CPD activity itself. The assessment validation process was detailed in 105 articles. In most cases, the authors examined the content with experts (n = 57) or pilot-tested the assessment (n = 50). We identified three themes: 1-satisfaction with assessment choices; 2-difficulties experienced during the administration of the assessment; and 3-complexity of the validation process.

DISCUSSION

Building on the adage "assessment drives learning," it is imperative that the CPD practices contribute to the intended learning and limit the unintended negative consequences of assessment. Our results suggest that validation processes must be considered and adapted within CPD contexts.

Defining chiropractic professional identity: A concept analysis

BACKGROUND

The concept of professional identity within chiropractic is often discussed and debated, however in the field to date, there is no formal definition of chiropractic professional identity (CPI). This article aims to create a coherent definition of CPI and to formalise the conceptual domains that may encompass it.

METHODS

Using the Walker and Avant (2005) process, a concept analysis methodology was employed to clarify the diffuse concept of CPI. This method initially involved selecting the concept (CPI), determining the aims and purpose of the analysis, identifying concept uses, and defining attributes. This was achieved from a critical literature review of professional identity across health disciplines. Chiropractic-related model, borderline and contrary cases were used to exemplify characteristics of CPI. The antecedents required to inform CPI, consequences of having, and ways to measure the concept of CPI were evaluated.

RESULTS

From the concept analysis data, CPI was found to encompass six broad attributes or domains: knowledge and understanding of professional ethics and standards of practice, chiropractic history, practice philosophy and motivations, the roles and expertise of a chiropractor, professional pride and attitude, and professional engagement and interaction behaviours. These domains were not mutually exclusive and may overlap.

CONCLUSION

A conceptual definition of CPI may bring together members and groups within the profession and promote intra-professional understanding across other disciplines. The CPI definition derived from this concept analysis is: 'A chiropractor's self-perception and ownership of their practice philosophies, roles and functions, and their pride, engagement, and knowledge of their profession'.

Optimizing the Learner's Role in Feedback: Development of a Feedback-Preparedness Online Application for Medical Students in the Clinical Setting

Feedback is an essential component of medical education, especially during clinical rotations. There is growing interest in learner-related factors that can optimize feedback's efficiency, including goal orientation, reflection, self-assessment, and emotional response. However, no mobile application or curriculum currently exists to specifically address those factors. This technical report describes the concept, design, and learner-based feedback of an innovative online application, available on mobile phones, developed to bridge this gap. Eighteen students in their third or fourth year of medical school provided comments on a pilot version of the application. The majority of learners deemed the module relevant, interesting, and helpful to guide reflection and self-assessment, therefore fostering better preparation before an upcoming feedback session. Minor improvements were suggested in terms of content and format. The learners' initial positive response supports further efforts to engage in validity and evaluation research. Future steps include modifying the mobile application based on learners' comments, evaluating its efficacy in a real clinical setting, and clarifying whether it is most beneficial for mid-rotation or end-of-rotation feedback sessions.

Professional identity formation: linking meaning to well-being

Trainee distress and burnout continue to be serious concerns for educational programs in medicine, prompting the implementation of numerous interventions. Although an expansive body of literature suggests that the experience of meaning at work is critical to professional wellbeing, relatively little attention has been paid to how this might be leveraged in the educational milieu. We propose that professional identity formation (PIF), the process by which trainees come to not only attain competence, but additionally to "think, act and feel" like physicians, affords us a unique opportunity to ground trainees in the meaningfulness of their work. Using the widely accepted tri-partite model of meaning, we outline how this process can contribute to wellbeing. We suggest strategies to optimize the influence of PIF on wellbeing, offering curricular suggestions, as well as ideas regarding the respective roles of communities of practice, teachers, and formative educational experiences. Collectively, these encourage trainees to act as intentional agents in the making of their novel professional selves, anchoring them to the meaningfulness of their work, and supporting their short and long-term wellbeing.

Confronting implicit bias toward patients: a scoping review of post-graduate physician curricula

BACKGROUND

Physicians' behavior may unknowingly be impacted by prejudice and thereby contribute to healthcare inequities. Despite increasingly robust data demonstrating physician implicit bias (The Office of Minority Health. Minority Population Profiles, 2021; COVID-19 Shines Light on Health Disparities, National Conference of State Legislatures 2021), the evidence behind how to change this with training programs remains unclear. This scoping review therefore reports on the implementation, outcomes, and characteristics of post-graduate physician implicit bias curricula.

METHODS

The authors conducted a literature review using scoping review methodology. They searched 7 databases in February and November 2020 for English-language academic and gray literature on implicit bias curricula for physicians at all levels of post-graduate training. Ten reviewers screened studies for eligibility independently, then extracted data from these studies and compiled it into a chart and analytical summary.

RESULTS

Of the 4,599 articles screened, this review identified 90 articles on implicit bias interventions for post-graduate physicians. Inductive data analysis revealed a spectrum of educational approaches, which were categorized int o 4 educational models called Competence, Skills-Based, Social Contact, and Critical Models. The most commonly reported strength was the interactive nature of the curricula (26%), and the most frequently identified challenges were related to time and resources available (53%). Half of the interventions discussed facilitator preparation, and the majority (62%) evaluated outcomes using pre and post self-assessments.

CONCLUSIONS

This review provides a comprehensive synthesis of the literature on physician implicit bias curricula. It is our goal that this supports medical educators in applying and improving aspects of these interventions in their own programs.

The education passport: connecting programmatic assessment across learning and practice

Competency-based medical education (CBME) shifts us from static assessment of learning to developmental assessment for learning. However, implementation challenges associated with CBME remain a major hurdle, especially after training and into practice. The full benefit of developmental assessment for learning over time requires collaboration, cooperation, and trust among learners, regulators, and the public that transcends each individual phase. The authors introduce the concept of an "Education Passport" that provides evidence of readiness to travel across the boundaries between undergraduate medical education, graduate medical education, and the expanse of practice. The Education Passport uses programmatic assessment, a process of collecting numerous low stakes assessments from multiple sources over time, judging these data using criterion-referencing, and enhancing this with coaching and competency committees to understand, process, and accelerate growth without end. Information in the Passport is housed on a cloud-based server controlled by the student/physician over the course of training and practice. These data are mapped to various educational frameworks such Entrustable Professional Activities or milestones for ease of longitudinal performance tracking. At each stage of education and practice the student/physician grants Passport access to all entities that can provide data on performance. Database managers use learning analytics to connect and display information over time that are then used by the student/physician, their assigned or chosen coaches, and review committees to maintain or improve performance. Global information is also collected and analyzed to improve the entire system of learning and care. Developing a true continuum that embraces performance and growth will be a long-term adaptive challenge across many organizations and jurisdictions and will require coordination from regulatory and national agencies. An Education Passport could also serve as an organizing tool and will require research and high-value communication strategies to maximize public trust in the work.

On embedding assessments of self-regulated learning into licensure activities in the health professions: a call to action

How well have healthcare professionals and trainees been prepared for the inevitable demands for new learning that will arise in their future? Given the rapidity with which 'core healthcare knowledge' changes, medical educators have a responsibility to audit whether trainees have developed the capacity to effectively self-regulate their learning. Trainees who engage in effective self-regulated learning (SRL) skillfully monitor and control their cognition, motivation, behaviour, and environment to adaptively meet demands for new learning. However, medical curricula rarely assess trainees' capacity to engage in these strategic processes. In this position paper, we argue for a paradigm shift toward assessing SRL more deliberately in undergraduate and postgraduate programs, as well as in associated licensing activities. Specifically, we explore evidence supporting an innovative blend of principles from the science on SRL, and on preparation for future learning (PFL) assessments. We propose recommendations for how program designers, curriculum developers, and assessment leads in undergraduate and postgraduate training programs, and in licensing bodies can work together to develop integrated assessments that measure how and how well trainees engage in SRL. Claims about lifelong learning in health professions education have gone unmatched by responsive curricular changes for far too long. Further neglecting these important competencies represents a disservice to medical trainees and a potential risk to the future patients they will care for.

Are there differences between SIMG surgeons and locally trained surgeons in Australia and New Zealand, as rated by colleagues and themselves?

BACKGROUND

Representation of specialist international medical graduates (SIMGs) in specific specialties such as surgery can be expected to grow as doctor shortages are predicted in the context of additional care provision for aging populations and limited local supply. Many national medical boards and colleges provide pathways for medical registration and fellowship of SIMGs that may include examinations and short-term training. There is currently very little understanding of how SIMGs are perceived by colleagues and whether their performance is perceived to be comparable to locally trained medical specialists. It is also not known how SIMGs perceive their own capabilities in comparison to local specialists. The aim of this study is to explore the relationships between colleague feedback and self-evaluation in the specialist area of surgery to identify possible methods for enhancing registration and follow-up training within the jurisdiction of Australia and New Zealand.

METHODS

Feedback from 1728 colleagues to 96 SIMG surgeons and 406 colleagues to 25 locally trained Fellow surgeons was collected, resulting in 2134 responses to 121 surgeons in total. Additionally, 98 SIMGs and 25 Fellows provided self-evaluation scores (123 in total). Questionnaire and data reliability were calculated before analysis of variance, principal component analysis and network analysis were performed to identify differences between colleague evaluations and self-evaluations by surgeon type.

RESULTS

Colleagues rated SIMGs and Fellows in the 'very good' to 'excellent' range. Fellows received a small but statistically significant higher average score than SIMGs, especially in areas dealing with medical skills and expertise. However, SIMGs received higher scores where there was motivation to demonstrate working well with colleagues. Colleagues rated SIMGs using one dimension and Fellows using three, which can be identified as clinical management skills, inter-personal communication skills and self-management skills. On self-evaluation, both SIMGs and Fellows gave themselves a significant lower average score than their colleagues, with SIMGs giving themselves a statistically significant higher score than Fellows.

CONCLUSIONS

Colleagues rate SIMGs and Fellows highly. The results of this study indicate that SIMGs tend to self-assess more highly, but according to colleagues do not display the same level of differentiation between clinical management, inter-personal and self-management skills. Further research is required to confirm these provisional findings and possible reasons for lack of differentiation if this exists. Depending on the outcome, possible support mechanisms can be explored that may lead to increased comparable performance with locally trained graduates of Australia and New Zealand in these three dimensions.

Musculoskeletal anatomy knowledge in Australian chiropractors

Anatomy is a key knowledge area in chiropractic and is formally offered in the undergraduate component of chiropractic education. There is the potential for loss of anatomy knowledge before the opportunity to apply it in a clinical setting. This study aimed to determine whether chiropractic clinicians retain a level of anatomy knowledge comparable to that of chiropractic students and to compare chiropractors' self-rating of their anatomical knowledge against an objective knowledge assessment tool. A previously validated multiple-choice test was utilized to measure retention of limb musculoskeletal (MSK) knowledge in Australian chiropractors. One hundred and one registered chiropractors completed the questionnaire and responses were scored, analyzed, and compared to scores attained by undergraduate and postgraduate chiropractic students who had previously completed the same questionnaire. The results indicated that practitioners retained their anatomy knowledge, with a significantly higher total mean score than the undergraduate group [total mean score = 36.5% (±SD 13.6%); P < 0.01] but not significantly different to the postgraduate group [total mean score = 52.2% (±SD 14.1%); P = 0.74]. There was a weak positive correlation between chiropractors' self-rated knowledge and test performance scores indicating the effectiveness of this Australian chiropractic group in self-assessing their anatomy knowledge. This study found that Australian chiropractors' knowledge of MSK anatomy was retained during the transition from university to clinical practice and they accurately evaluated their own test performance.

Affecting Change: Enhancing Feedback Interactions with Radiology Trainees

Feedback is a critical part of the learning process and is a valuable tool to empower adult learners. Modern feedback theory places the learner at the center of the feedback encounter. Individual and institutional barriers to effective giving and receiving of feedback can be overcome through education and attention to the form and content of feedback. We review the elements of effective feedback and address issues of framing, environmental, and social factors which aid in providing psychological safety and trust, as necessary elements to create a culture of feedback in radiology training programs. We provide practical strategies to empower learners with the necessary skills to solicit, receive, and reflect on feedback.

Evaluating the impact of integrating SMART goal setting in preceptor development using the Habits of Preceptors Rubric

DISCLAIMER

In an effort to expedite the publication of articles, AJHP is posting manuscripts online as soon as possible after acceptance. Accepted manuscripts have been peer-reviewed and copyedited, but are posted online before technical formatting and author proofing. These manuscripts are not the final version of record and will be replaced with the final article (formatted per AJHP style and proofed by the authors) at a later time.

PURPOSE

The impact of goal setting in pharmacy preceptor development was evaluated using the Habits of Preceptors Rubric (HOP-R), a criterion-referenced assessment developed to assess, quantify, and demonstrate growth across 11 preceptor habits.

METHODS

This study retrospectively evaluated initial and follow-up survey responses from the 2019-2020 Clinician Educators Program cohort at Midwestern University College of Pharmacy, Glendale Campus. Enrollees in this teaching and learning curriculum (TLC) were invited to assess their precepting habits using the HOP-R after attending the first seminar and again toward the end of the longitudinal program. Using online surveys, participants rated their precepting capabilities as developing, proficient, accomplished, or master level for each habit. In the initial survey, each participant selected a habit of focus for deliberate development and established an individualized goal using the specific, measurable, achievable, relevant, and time-bound (SMART) framework. In the follow-up survey, participants indicated their satisfaction with and progress toward accomplishing their precepting goal.

RESULTS

Initial survey results from 55 study participants identified developing as the most frequently self-reported habit level (53%; n = 605 ratings), while master was the least frequently reported (1%). In the follow-up assessment, accomplished (45%) was the most frequently self-reported habit level, while master (5%) remained the least reported. The cohort reported a median progress in SMART goal accomplishment of 69% (range, 12% to 100%; n = 54).

CONCLUSION

Preceptors and resident pharmacists reported perceived advancement in precepting capabilities within the conceptual framework of the HOP-R encompassing both their self-selected habit of focus and adjacent habits while enrolled in a TLC. SMART goals facilitated qualitative and quantitative assessment of development.

An Article-Based Format for Medical Specialty and Subspecialty Maintenance of Certification

INTRODUCTION

This article describes an article-based alternative for maintenance of certification that the American Board of Psychiatry and Neurology developed and began pilot testing in 2019. The rationale for and components of the pilot program are presented along with data on participant performance and feedback from the first year of implementation in three primary specialties (neurology, child neurology, and psychiatry) and one subspecialty (child and adolescent psychiatry).

METHODS

Evaluation of the pilot program was guided by a widely used validity framework. Data were collected that addressed the five categories of validity evidence: content, response process, internal structure, relation to other variables, and consequences.

RESULTS

Enrollment ranged from 66.7% for psychiatrists to 75.3% for child neurologists. For the 2019 cohort, the pass rates ranged from 92.6% for child and adolescent psychiatry to 98.7% for neurology, and very small numbers of diplomates failed or did not complete the process. For psychiatrists, there was a modest, but significant, relationship between performance on previous and subsequent maintenance of certification examinations. Ninety percent or more agreed that: the articles were easy to access and helpful to their practices; the mini-tests were a fair assessment of their understanding of the articles; and their test-taking experience was satisfactory.

DISCUSSION

Most eligible diplomates participated in the article-based pilot project, and they strongly preferred this format to the traditional multiple-choice examinations. Most important, the pilot was perceived to be a meaningful and relevant learning activity that had a positive effect on patient care.

Value of a Growth Mindset: Improving Endoscopy Coaching and Mentoring

The teaching of endoscopic procedures to trainees has radically changed from the previous "see one, do one, teach one" approach to the method termed "competency-based medical education" (CBME), which has transformed endoscopic training. Successful implementation of a CBME program requires learners to direct their own learning. Learners with a growth approach are more likely to thrive in such a system since they are learning-oriented and not performance-oriented. This method dictates that endoscopy mentors take a longer-term and broader view of the development of the trainee and are vital to cultivate a growth mindset in learners, while an endoscopy coach focuses on helping trainees apply principles of mastery learning and deliberate practice. The goal of the authors was to describe some principles of effective coaching and mentoring methods as applied to endoscopy training.

The Learning Loop: Conceptualizing Just-in-Time Faculty Development

BACKGROUND

As technology advances, the gap between learning and doing continues to close-especially for frontline academic faculty and clinician educators. For busy clinician faculty members, it can be difficult to find time to engage in skills and professional development. Competing interests between clinical care and various forms of academic work (e.g., research, administration, education) all create challenges for traditional group-based and/or didactic faculty development.

METHODS

The authors engaged in a synthetic narrative review of literature from several unrelated fields: learning technologies, medical education/health professions education, general/higher education. The aim for this review was to synthesize this pre-existing literature to propose a new conceptual model.

RESULTS

The authors propose a new conceptual model, the Just-In-Time Learning Loop, to guide the development of online faculty development for just-in-time delivery.

CONCLUSIONS

The Just-In-Time Learning Loop is a new conceptual framework that may be of use to those engaging in online, digital learning design. Faculty developers, especially in emergency medicine, can integrate leading concepts from the technology-enhanced learning field (e.g., microlearning, micro-credentialing, badging) to create new types of learning experiences for their end-users.

Confidence-competence alignment and the role of self-confidence in medical education: A conceptual review

CONTEXT

There have been significant advances in competency-based medical education (CBME) within health professions education. While most of the efforts have focused on competency, less attention has been paid to the role of confidence as a factor in preparing for practice. This paper seeks to address this deficit by exploring the role of confidence and the calibration of confidence with regard to competence.

METHODS

This paper presents a conceptual review of confidence and the calibration of confidence in different medical education contexts. Building from an initial literature review, the authors engaged in iterative discussions exploring divergent and convergent perspectives, which were then supplemented with targeted literature reviews. Finally, a stakeholder consultation was conducted to situate and validate the provisional findings.

RESULTS

A series of axioms were developed to guide perceptions and responses to different states of confidence in health professionals: (a) confidence can shape how we act and is optimised when it closely corresponds to reality; (b) self-confidence is task-specific, but also inextricably influenced by the individual self-conceptualisation, the surrounding system and society; (c) confidence is shaped by many external factors and the context of the situation; (d) confidence must be considered in conjunction with competence and (e) the confidence-competence ratio (CCR) changes over time. It is important to track learners' CCRs and work with them to maintain balance.

CONCLUSION

Confidence is expressed in different ways and is shaped by a variety of modifiers. While CBME primarily focuses on competency, proportional confidence is an integral component in ensuring safe and professional practice. As such, it is important to consider both confidence and competence, as well as their relationship in CBME. The CCR can serve as a key construct in developing mindful and capable health professionals. Future research should evaluate strategies for assessing CCR, identify best practices for teaching confidence and guiding self-calibration of CCR and explore the role of CCR in continuing professional development for individuals and teams.

Assessing learner engagement with virtual educational events: Development of the Virtual In-Class Engagement Measure (VIEM)

BACKGROUND

The COVID-19 pandemic has necessitated virtual education, but effects on learner engagement are unknown. We developed a virtual in-class engagement measure (VIEM) to assess learner engagement in online surgical education events.

METHODS

Using the STROBE, an observer collected tool to document student engagement, as a template an ASE committee workgroup developed the VIEM. The VIEM had two parts: observer assessment and learner self-assessment of engagement. Trained observers collected engagement data from two institutions using the VIEM. Surgical attendings, fellows and residents were observed during virtual learning events. Educator attitudes towards online teaching were also assessed via survey.

RESULTS

22 events with 839 learners were observed. VIEM distinguished between sessions with low and high engagement. 20% of learners pretended to participate. Half of instructors were comfortable with virtual teaching, but only 1/3 believed was as effective as in-person. 2/3 of teachers believed video learners were more engaged than audio learners.

CONCLUSIONS

Virtual platforms do not automatically translate into increased engagement. Standard tools such as VIEM may help with assessment of engagement during virtual education.

Re-envisioning paradigms of education: towards awareness, alignment, and pluralism

In this article we introduce a synthesis of education "paradigms," adapted from a multi-disciplinary body of literature and tailored to health professions education (HPE). Each paradigm involves a particular perspective on the purpose of education, the nature of knowledge, what knowledge is valued and included in the curriculum, what it means to learn and how learning is assessed, and the roles of teachers and learners in the learning process. We aim to foster awareness of how these different paradigms look in practice and to illustrate the importance of alignment between teaching, learning and assessment practices with paradigmatic values and assumptions. Finally, we advocate for a pluralistic approach that purposefully and meaningfully integrates paradigms of education, enhancing our ability to drive quality in HPE.

Medical Students' Perspectives on Entrustment Decision Making in an Entrustable Professional Activity Assessment Framework: A Secondary Data Analysis

PURPOSE

Research on how entrustment decisions are made identifies 5 influential factors (supervisor, trainee, supervisor-trainee relationship, context, task). However, this literature primarily represents the perspective of supervisors in graduate medical education and is conducted outside of an assessment framework where entrustment decisions have consequences for trainees and for patients. To complement the literature, the authors explored how medical students in a pilot program that used an entrustable professional activity (EPA) assessment framework perceived factors influencing entrustment decisions.

METHOD

The authors conducted a secondary analysis of qualitative data from program evaluation of a pilot project using an EPA assessment framework to advance students through their curriculum and into residency. Data were derived from 8 focus groups conducted over 4 years (2015-2018) with 27 students who shared their experience of learning and supervision in the project. Sensitized by the entrustment literature, the authors coded and analyzed focus group transcripts according to principles of thematic analysis.

RESULTS

Components of the trainee and supervisor-trainee relationship factors predominated students' perceptions of entrustment decisions. Students directed their own learning by asking for feedback, seeking opportunities to engage in learning, sharing limitations of their knowledge with supervisors, and pushing supervisors to recalibrate assessments when appropriate. Students' agentic actions were facilitated by longitudinal supervisor-trainee relationships wherein they felt comfortable asking for help and built confidence in patient care. Students mentioned components of other factors that influenced entrustment decisions (supervisor, clinical task, clinical context), but did so less frequently and from a nonagentic vantage point.

CONCLUSIONS

Students' perspectives on entrustment decisions can be derived from their views on learning and supervision in an EPA assessment framework. Their perspectives complement the literature by highlighting students' agentic actions to influence entrustment decisions and promotion of agentic action through practices incorporating longitudinal supervisor-trainee relationships.

Evaluation of an eLearning teaching innovation to assist clinical radiation therapy educators in the provision of student feedback

BACKGROUND

Clinical placement is an integral part of the Radiation Therapy undergraduate programme. Feedback and formative assessment during clinical placement are regarded as key to developing clinical skills and competencies. Students regularly report dissatisfaction with the feedback process while clinical educators report heavy clinical workloads and a lack of guidance on feedback mechanisms as barriers to providing meaningful student feedback.

METHODS

An eLearning teaching intervention was developed to support radiation therapists in the provision of student feedback in the clinic. Thematic analysis was used to report attitudes to feedback and feedback practices collected in a pre and a post intervention evaluation.

RESULTS

30 radiation therapists completed the module and pre and post intervention evaluations. Prior to taking the module just over half of respondents stated that they offered regular and on-going feedback throughout the student's placement. Positive attitudes to feedback were reported. Following completion of the eLearning tool respondents reported a higher level of confidence in the provision of student feedback and almost 70% said the module had changed how they would approach the feedback process by using feedback models in the future.

DISCUSSION

Good and timely feedback is essential and allows a student opportunity to improve prior to the end of the placement. It also teaches students how to self-assess and self-reflect - skills that they can use in continuous professional development after they graduate. Radiation therapists appreciate the structure that using a model in the feedback process offers.

CONCLUSION

This eLearning teaching intervention was received favourably by radiation therapists who are key to creating a culture of feedback in the clinical environment that will facilitate students in becoming competent healthcare professionals.

Educational needs of fertility healthcare professionals using ART: a multi-country mixed-methods study

RESEARCH QUESTION

What are the most pressing educational needs of fertility healthcare professionals using assisted reproductive technologies (ART)?

DESIGN

This mixed-methods study combined qualitative interviews with quantitative surveys. Participants included physicians and nurses specialized in reproductive endocrinology or in obstetrics/gynaecology, and laboratory specialists, with a minimum of 3 years of experience, practising in Australia, Brazil, Canada, China, France, Germany, India, Italy, Japan, Mexico, Spain or the UK. Maximum variation purposive sampling was used to ensure a mix of experience and settings. Interviews were transcribed and coded through thematic analysis. Quantitative data were analysed using frequency tables, cross-tabulations and chi-squared tests to compare results by reimbursement context.

RESULTS

A total of 535 participants were included (273 physicians, 145 nurses and 117 laboratory specialists). Knowledge gaps, skills gaps and attitude issues were identified in relation to: (i) ovarian stimulation (e.g. knowledge of treatments and instruction protocols for ovarian stimulation), (ii) embryo culture and cryopreservation/vitrification (e.g. diverging opinions on embryo freezing, (iii) embryo assessment (e.g. performing genetic testing), (iv) support of luteal phase and optimizing pregnancy outcomes (e.g. knowledge of assessment methods for endometrial receptivity), and (v) communication with patients (e.g. reluctance to address emotional distress).

CONCLUSIONS

This descriptive, exploratory study corroborates previously reported gaps in fertility care and identifies potential causes of these gaps. Findings provide evidence to inform educational programmes for healthcare professionals who use ART in their practice and calls for the development of case-based education and interprofessional training programmes to improve care for patients with fertility issues.

Medical student self-assessment as emergency medicine residency applicants

BACKGROUND

Emergency medicine (EM) applicants are encouraged to consider their own "competitiveness" when deciding on the number of applications to submit. Program directors rank the Standardized Letter of Evaluation (SLOE) as the most important factor when reviewing an applicant. Accurate insight into how clinical performance is reflected on the SLOE could improve medical students' ability to gauge their own competitiveness.

OBJECTIVE

This study aims to determine the accuracy of students' self-assessment by SLOE evaluation measures when compared to the SLOE completed by faculty after their EM clerkship.

METHODS

Participants of this multicenter study included fourth-year medical students who had completed their EM clerkship and were applying to EM residency. Students completed a modified SLOE to reflect rankings they believed they would receive on their official SLOE. Additionally, students completed a survey assessing their knowledge of the SLOE, their perception of feedback during the clerkship, and their self-perceived competitiveness as an EM applicant. Correlation between the rankings on the student-completed SLOE and the official SLOE was analyzed using the Kendall correlation.

RESULTS

Of the 49 eligible students, 42 (85.7%) completed the study. The correlation between scores on the student-completed and official SLOE were significantly low (r < 0.68) for each item. The majority of students agreed that they were satisfied by the quantity and quality of feedback they received (31/42, 73.8%). Few students agreed that they knew how many applications to submit to ensure a match in EM (7/42, 16.7%).

CONCLUSION

This study demonstrates that students did not accurately predict their rankings on the official SLOE at the end of an EM rotation and had little insight into their competitiveness as an applicant. These findings highlight opportunities to mitigate the burden on students and programs caused by the increasing number of applications per applicant. Further research is needed as to whether strategies to increase insight into competitiveness are effective.

Becoming a deliberately developmental organization: Using competency based assessment data for organizational development

Medical education is situated within health care and educational organizations that frequently lag in their use of data to learn, develop, and improve performance. How might we leverage competency-based medical education (CBME) assessment data at the individual, program, and system levels, with the goal of redefining CBME from an initiative that supports the development of physicians to one that also fosters the development of the faculty, administrators, and programs within our organizations? In this paper we review the Deliberately Developmental Organization (DDO) framework proposed by Robert Kegan and Lisa Lahey, a theoretical framework that explains how organizations can foster the development of their people. We then describe the DDO's conceptual alignment with CBME and outline how CBME assessment data could be used to spur the transformation of health care and educational organizations into digitally integrated DDOs. A DDO-oriented use of CBME assessment data will require intentional investment into both the digitalization of assessment data and the development of the people within our organizations. By reframing CBME in this light, we hope that educational and health care leaders will see their investments in CBME as an opportunity to spur the evolution of a developmental culture.

Knowledge to action: a scoping review of approaches to educate primary care providers in the identification and management of routine sleep disorders

STUDY OBJECTIVES

The referral burden on healthcare systems for routine sleep disorders could be alleviated by educating primary care providers (PCPs) to diagnose and manage patients with sleep health issues. This requires effective professional education strategies and resources. This scoping review examined the literature on existing approaches to educate PCPs in sleep health management.

METHODS

A comprehensive literature search was conducted across eight databases to identify citations describing the education of PCPs in diagnosing and managing sleep disorders, specifically insomnia and sleep apnea. A conceptual framework, developed from the knowledge-to-action cycle, was used to analyze citations from a knowledge translation perspective.

RESULTS

Searches identified 616 unique citations and after selection criteria were applied, 22 reports were included. Reports spanning 38 years were analyzed using components of the knowledge-to-action cycle to understand how educational interventions were designed, developed, implemented, and evaluated. Interventions involved didactic (32%), active (18%) and blended (41%) approaches, using face-to-face (27%), technology-mediated (45%) and multimodal (5%) delivery. Educational effectiveness was assessed in 73% of reports, most commonly using a pre/post questionnaire (41%).

CONCLUSIONS

While this scoping review has utility in describing existing educational interventions to upskill PCPs to diagnose and manage sleep disorders, the findings suggest that interventions are often developed without explicitly considering the evidence of best educational practice. Future interventional designs may achieve greater sustained effectiveness by considering characteristics of the target audience, the pedagogical approaches best suited to its needs, and any environmental drivers and barriers that might impede the translation of evidence into practice.

Myth or Reality: Self-Assessment Is Central to Effective Curriculum in Anatomical Pathology Graduate Medical Education

Self-assessment, a personal evaluation of one’s professional attributes and abilities against a perceived norm, has frequently been cited as a necessary component of self-directed learning and the maintenance of competency within regulated health professions, including the medical professions. However, education research literature has consistently shown uninformed personal global assessment of performance to be inaccurate in a variety of contexts, and have limited value in a workplace-based curriculum. Incorporating known standards of performance with internal and external data on the performance improves a learner’s ability to accurately self-assess. Selecting content suitable for self-assessment, providing explicit assessment standards, encouraging feedback-seeking behaviors, supporting a growth mindset, and providing quality feedback in a supportive context are all strategies that can support learner self-assessment, learner engagement in reflection, and action on feedback in Anatomical Pathology graduate medical education.

Image interpretation: Learning analytics-informed education opportunities

OBJECTIVES

Using a sample of pediatric chest radiographs (pCXR) taken to rule out pneumonia, we obtained diagnostic interpretations from physicians and used learning analytics to determine the radiographic variables and participant review processes that predicted for an incorrect diagnostic interpretation.

METHODS

This was a prospective cross-sectional study. A convenience sample of frontline physicians with a range of experience levels interpreted 200 pCXR presented using a customized online radiograph presentation platform. Participants were asked to determine absence or presence (with respective location) of pneumonia. The pCXR were categorized for specific image-based variables potentially associated with interpretation difficulty. We also generated heat maps displaying the locations of diagnostic error among normal pCXR. Finally, we compared image review processes in participants with higher versus lower levels of clinical experience.

RESULTS

We enrolled 83 participants (20 medical students, 40 postgraduate trainees, and 23 faculty) and obtained 12,178 case interpretations. Variables that predicted for increased pCXR interpretation difficulty were pneumonia versus no pneumonia (β = 8.7, 95% confidence interval [CI] = 7.4 to 10.0), low versus higher visibility of pneumonia (β = -2.2, 95% CI = -2.7 to -1.7), nonspecific lung pathology (β = 0.9, 95% CI = 0.40 to 1.5), localized versus multifocal pneumonia (β = -0.5, 95% CI = -0.8 to -0.1), and one versus two views (β = 0.9, 95% CI = 0.01 to 1.9). A review of diagnostic errors identified that bony structures, vessels in the perihilar region, peribronchial thickening, and thymus were often mistaken for pneumonia. Participants with lower experience were less accurate when they reviewed one of two available views (p < 0.0001), and accuracy of those with higher experience increased with increased confidence in their response (p < 0.0001).

CONCLUSIONS

Using learning analytics, we identified actionable learning opportunities for pCXR interpretation, which can be used to allow for a customized weighting of which cases to practice. Furthermore, experienced-novice comparisons revealed image review processes that were associated with greater diagnostic accuracy, providing additional insight into skill development of image interpretation.

Making Decision-Making Visible-Teaching the Process of Evaluating Interventions

Significant efforts in the past decades to teach evidence-based practice (EBP) implementation has emphasized increasing knowledge of EBP and developing interventions to support adoption to practice. These efforts have resulted in only limited sustained improvements in the daily use of evidence-based interventions in clinical practice in most health professions. Many new interventions with limited evidence of effectiveness are readily adopted each year-indicating openness to change is not the problem. The selection of an intervention is the outcome of an elaborate and complex cognitive process, which is shaped by how they represent the problem in their mind and is mostly invisible processes to others. Therefore, the complex thinking process that support appropriate adoption of interventions should be taught more explicitly. Making the process visible to clinicians increases the acquisition of the skills required to judiciously select one intervention over others. The purpose of this paper is to provide a review of the selection process and the critical analysis that is required to appropriately decide to trial or not trial new intervention strategies with patients.

Clinical supervision in general practice training: the interweaving of supervisor, trainee and patient entrustment with clinical oversight, patient safety and trainee learning

Australian general practice trainees typically consult with patients from their first week of training, seeking in-consultation supervisory assistance only when not sufficiently confident to complete patient consultations independently. Trainee help-seeking plays a key role in supervisor oversight of trainee consultations. This study used focus groups and interviews with general practice supervisors to explore their approaches to trainee help-seeking and in-consultation supervision. Supervisor approaches are discussed under three themes: establishing a help-seeking culture; perceptions of in-consultation assistance required; and scripts for help provision. Within these themes, three interwoven entrustment processes were identified: supervisor entrustment; trainee self-entrustment; and 'patient entrustment' (patient confidence in the trainee's clinical management). Entrustment appears to develop rapidly, holistically and informally in general practice training, partly in response to workflow pressure and time constraints. Typical supervisor scripts and etiquette for help-provision involve indirect, soft correction strategies to build trainee self-entrustment. These scripts appear to be difficult to adapt appropriately to under-performing trainees. Importantly, supervisor scripts also promote patient entrustment, increasing the likelihood of patients returning to the trainee and training practice for subsequent review, which is a major mechanism for ensuring patient safety in general practice. Theories of entrustment in general practice training must account for the interplay between supervisor, trainee and patient entrustment processes, and work-related constraints. Gaps between entrustment as espoused in theory, and entrustment as enacted, may suggest limitations of entrustment theory when extended to the general practice context, and/or room for improvement in the oversight of trainee consultations in general practice training.

An innovative pharmacology curriculum for medical students: promoting higher order cognition, learner-centered coaching, and constructive feedback through a social pedagogy framework

BACKGROUND

Ongoing developments in medical education recognize the move to curricula that support self-regulated learning processes, skills of thinking, and the ability to adapt and navigate uncertain situations as much as the knowledge base of learners. Difficulties encountered in pursuing this reform, especially for pharmacology, include the tendency of beginner learners not to ask higher-order questions and the potential incongruency between creating authentic spaces for self-directed learning and providing external expert guidance. We tested the feasibility of developing, implementing, and sustaining an innovative model of social pedagogy as a strategy to address these challenges.

METHODS

Constructivism, communities of practice, and networked learning theory were selected as lenses for development of the model. Three hundred sixty-five first-year medical students participated between 2014 and 2018; they were introduced to pharmacodynamics and pharmacokinetics via 15 online modules that each included: learning objectives, a clinical vignette, teaching video, cumulative concept map, and small group wiki assignment. Five-person communities organized around the 15 wiki assignments were a key component where learners answered asynchronous, case-based questions that touched iteratively on Bloom's cognitive taxonomy levels. The social pedagogy model's wiki assignments were explored using abductive qualitative data analysis.

RESULTS

Qualitative analysis revealed that learners acquired and applied a conceptual framework for approaching pharmacology as a discipline, and demonstrated adaptive mastery by evaluating and interacting competently with unfamiliar drug information. Learners and faculty acquired habits of self-directed assessment seeking and learner-centered coaching, respectively; specifically, the model taught learners to look outward to peers, faculty, and external sources of information for credible and constructive feedback, and that this feedback could be trusted as a basis to direct performance improvement. 82-94% of learners rated the social pedagogy-based curriculum valuable.

CONCLUSIONS

This social pedagogy model is agnostic with regard to pharmacology and type of health professional learner; therefore, we anticipate its benefits to be transferable to other disciplines.

Are Self-Assessment and Peer Assessment of Added Value in Training Complex Pediatric Surgical Skills?

INTRODUCTION

 Self-assessment aids "reflection-before-practice," which is expected to result in a better understanding of one's strengths and weaknesses and consequently a better overall performance. This is, however, rarely used in surgical training. This study aims to evaluate the correlation between self-, peer-, and expert assessment on surgical skills of pediatric surgical trainees.

MATERIALS AND METHODS

 A competency assessment tool for the posterior sagittal anorectoplasty (CAT-PSARP) was previously developed and validated. During international hands-on pediatric colorectal workshops in 2019 and 2020, participants practiced the PSARP on an inanimate anorectal malformation model. They were assisted by a peer and assessed by two independent expert observers, using the CAT-PSARP. After the training session, both self- and peer assessment were completed, using the same CAT-PSARP.

RESULTS

 A total of 79 participants were included. No correlation was found between the overall CAT-PSARP scores of the expert observers and the self-assessment (r = 0.179, p = 0.116), while a weak correlation was found between experts and peer assessment (r = 0.317, p = 0.006). When comparing the self-assessment scores with peer assessment, a moderate correlation was found for the overall performance score (r = 0.495, p < 0.001). Additionally, the participants who were first to perform the procedure scored significantly better than those who trained second on the overall performance (mean 27.2 vs. 24.4, p < 0.001).

CONCLUSION

 Participants, peers, and experts seemed to have a unique view on the performance during training because there was little correlation between outcomes of the trainees and the experts. Self-assessment may be useful for reflection during training; however, expert assessment seems to be essential for assessment of surgical skills.

Evaluation of a Novel Laparoscopic Cholecystectomy Curriculum With the Use of Animal Models and Live Operating

BACKGROUND

Despite wide recognition of the benefits of simulation training, there is no established model for the teaching of laparoscopic cholecystectomy. The authors developed a replicable, intensive 2-day proficiency-based curriculum, to include simulation and live operating, designed to be practical to both attend and organize. The primary objective of this study was to evaluate this curriculum for improved procedural confidence, measured by participant self-evaluation. Secondary outcomes were objective improvement in technical skills in accordance with the Objective Structured Assessment of Technical Skills (OSATS) scale and trainee self-assessment scores.

METHODS

The course consisted of lectures, operating on a sheep hepatobiliary model through a laparoscopic box trainer, and live operating on female patient volunteers. It was attended by eight junior registrars. Precourse data collated included demographic information, experience, and procedural confidence scores using a visual analog scale. Performance on an animal model and live patient was assessed by experts using the OSATS score. Procedural confidence was re-evaluated after each task, as well as self-assessment of speed, accuracy, and overall performance.

RESULTS

Procedural confidence scores improved by a mean of 12% (P < 0.001). All trainees demonstrated sustained objective improvement in technical skills (P < 0.001). The overall mean OSATS score increased by 18%. Significant improvement was observed after performing the procedure on an animal model (P < 0.001); however, no further significant improvement was observed with live operating. No significant difference was found on trainee self-assessment scores, in any category.

CONCLUSIONS

The study describes a successful curriculum model for the teaching of laparoscopic cholecystectomy, to include procedural and technical skill acquisition, in addition to the refinement and development of procedural confidence. Importantly, this was carried out in a safe environment with direct transferability to the operating theater.

Simulation training for emergency skills: effects on ICU fellows' performance and supervision levels

BACKGROUND

The application of manual emergency skills is essential in intensive care medicine. Simulation training on cadavers may be beneficial. The aim of this study was to analyze a skill-training aiming to enhance ICU-fellows´ performance.

METHODS

A skill-training was prepared for chest tube insertion, pericardiocentesis, and cricothyroidotomy. Supervision levels (SL) for entrustable professional activities (EPA) were applied to evaluate skill performance. Pre- and post-training, SL and fellows´ self- versus consultants´ external assessment was compared. Time on skill training was compared to conventional training in the ICU-setting.

RESULTS

Comparison of pre/post external assessment showed reduced required SL for chest tube insertion, pericardiocentesis, and cricothyroidotomy. Self- and external assessed SL did not significantly correlate for pre-training/post-training pericardiocentesis and post-training cricothyroidotomy. Correlations were observed for self- and external assessment SL for chest tube insertion and pre-assessment for cricothyroidotomy. Compared to conventional training in the ICU-setting, chest tube insertion training may further be time-saving.

CONCLUSIONS

Emergency skill training separated from a daily clinical ICU-setting appeared feasible and useful to enhance skill performance in ICU fellows and may reduce respective SL. We observed that in dedicated skill-training sessions, required time resources would be somewhat reduced compared to conventional training methods.

Receiving Real-Time Clinical Feedback: A Workshop and OSTE Assessment for Medical Students

BACKGROUND

Many programs designed to improve feedback to students focus on faculty's ability to provide a safe learning environment, and specific, actionable suggestions for improvement. Little attention has been paid to improving students' attitudes and skills in accepting and responding to feedback effectively. Effective "real-time" feedback in the clinical setting is dependent on both the skill of the teacher and the learner's ability to receive the feedback. Medical students entering their clinical clerkships are not formally trained in receiving feedback, despite the significant amount of feedback received during this time.

METHODS

We developed and implemented a one-hour workshop to teach medical students strategies for effectively receiving and responding to "real-time" (formative) feedback in the clinical environment. Subjective confidence and skill in receiving real-time feedback were assessed in pre- and post-workshop surveys. Objective performance of receiving feedback was evaluated before and after the workshop using a simulated feedback encounter designed to re-create common clinical and cognitive pitfalls for medical students, called an objective structured teaching exercise (OSTE).

RESULTS

After a single workshop, students self-reported increased confidence (mean 6.0 to 7.4 out of 10, P<0.01) and skill (mean 6.0 to 7.0 out of 10, P=0.10). Compared to pre-workshop OSTE scores, post-workshop OSTE scores objectively measuring skill in receiving feedback were also significantly higher (mean 28.8 to 34.5 out of 40, P=0.0131).

CONCLUSION

A one-hour workshop dedicated to strategies in receiving real-time feedback may improve effective feedback reception as well as self-perceived skill and confidence in receiving feedback. Providing strategies to trainees to improve their ability to effectively receive feedback may be a high-yield approach to both strengthen the power of feedback in the clinical environment and enrich the clinical experience of the medical student.