Beyond 'driving': The relationship between assessment, performance and learning

Does 'summative' count? The influence of the awarding of study credits on feedback use and test-taking motivation in medical progress testing

Despite the increasing implementation of formative assessment in medical education, its' effect on learning behaviour remains questionable. This effect may depend on how students value formative, and summative assessments differently. Informed by Expectancy Value Theory, we compared test preparation, feedback use, and test-taking motivation of medical students who either took a purely formative progress test (formative PT-group) or a progress test that yielded study credits (summative PT-group). In a mixed-methods study design, we triangulated quantitative questionnaire data (n = 264), logging data of an online PT feedback system (n = 618), and qualitative interview data (n = 21) to compare feedback use, and test-taking motivation between the formative PT-group (n = 316), and the summative PT-group (n = 302). Self-reported, and actual feedback consultation was higher in the summative PT-group. Test preparation, and active feedback use were relatively low and similar in both groups. Both quantitative, and qualitative results showed that the motivation to prepare and consult feedback relates to how students value the assessment. In the interview data, a link could be made with goal orientation theory, as performance-oriented students perceived the formative PT as not important due to the lack of study credits. This led to low test-taking effort, and feedback consultation after the formative PT. In contrast, learning-oriented students valued the formative PT, and used it for self-study or self-assessment to gain feedback. Our results indicate that most students are less motivated to put effort in the test, and use feedback when there are no direct consequences. A supportive assessment environment that emphasizes recognition of the value of formative testing is required to motivate students to use feedback for learning.

'Having skin in the game': guiding principles for incorporating moulage into OSCEs

BACKGROUND

Dermatological conditions are a common reason for patients to seek healthcare advice. However, they are often under-represented in Objective Structured Clinical Examinations (OSCEs). Given the visual nature of skin conditions, simulation is suited to recreate such skin conditions in assessments such as OSCEs. One such technique often used in simulation is moulage-the art and science of using special effects make-up techniques to replicate a wide range of conditions on Simulated Participants or manikins. However, the contextual nature of OSCEs places additional challenges compared to using moulage in more general forms of simulated-based education.

MAIN BODY

OSCEs are high-stakes assessments and require standardisation across multiple OSCE circuits. In addition, OSCEs tend to have large numbers of candidates, so moulage needs to be durable in this context. Given the need to expand the use of moulage in OSCE stations and the unique challenges that occur in OSCEs, there is a requirement to have guiding principles to inform their use and development.

CONCLUSION

Informed by evidence, and grounded in experience, this article aims to provide practical tips for health profession education faculty on how best to optimise the use of moulage in OSCEs. We will describe the process of designing an OSCE station, with a focus on including moulage. Secondly, we will provide a series of important practice points to use moulage in OSCEs-and encourage readers to integrate them into their day-to-day practice.

Curriculum Indigenization in oral health professions' education worldwide: A scoping review

OBJECTIVE

To explore the literature on Indigenous content within the oral health professions' education curricula.

METHODS

This scoping review included all types of literature on oral health care educational programs on Indigenous content, following the JBI (Joanna Briggs Institute) methodology. An initial search using "Indigenous," "education," and "oral health" as keywords informed a full search strategy for MEDLINE, CINAHL, Embase, Scopus, ERIC, EPPI, MedEdPORTAL, Google Scholar, ProQuest Dissertations and Theses Global, Australian Government Department of Health, and Australian Indigenous HealthInfoNet. The search included literature available until November 1, 2023, irrespective of language. Two reviewers independently screened the studies, and data were extracted and presented in tabular and narrative summary formats.

RESULTS

A total of 948 records were identified, and 101 studies were chosen for full-text review. Twenty-three studies met the criteria for data extraction. Of all studies, 95.6% were published between 2007 and 2021, mostly from Australia and New Zealand. The most frequently covered content included Indigenous culture, followed by history, Indigenous oral health, and Indigenous Peoples' health. Rural and clinical placements were the most employed delivery methods, and evaluation surveys were the most employed assessment technique. Barriers to delivering an Indigenous curriculum included students' disinterest and limited interaction with Indigenous communities, while facilitators included cultural immersion and supportive mentorship.

CONCLUSION

Despite progress in integrating Indigenous content into oral health education, challenges persist. Prioritizing Indigenous perspectives, community partnerships, and standardized assessment tools is needed. Future research should focus on long-term impacts and best practices for Indigenous curriculum development and delivery.

Twelve tips for developing healthcare learners' uncertainty tolerance

BACKGROUND

Uncertainty is pervasive throughout healthcare practice. Uncertainty tolerance (i.e. adaptively responding to perceived uncertainty) is considered to benefit practitioner wellbeing, encourage person-centred care, and support judicious healthcare resource utilisation. Accordingly, uncertainty tolerance development is increasingly referenced within training frameworks. Practical approaches to support healthcare learners' uncertainty tolerance development, however, are lacking.

AIMS

Drawing on findings across the literature, and the authors' educational experiences, twelve tips for promoting healthcare learners' uncertainty tolerance were developed.

RESULTS

Tips are divided into 1. Tips for Learners, 2. Tips for Educators and Supervisors, and 3. Tips for Healthcare Education Institutions and Systems. Each tip summarises relevant research findings, alongside applications to educational practice.

CONCLUSIONS

Approaches to developing uncertainty tolerance balance factors supporting learners through uncertain experiences, with introducing challenges for learners to further develop uncertainty tolerance. These tips can reassure healthcare education stakeholders that developing learner uncertainty tolerance, alongside core knowledge, is achievable.

Assessment-Seeking Strategies:

Phenomenon: Competency-based medical education (CBME) relies on workplace-based assessment (WBA) to generate formative feedback (assessment for learning-AfL) and make inferences about competence (assessment of learning-AoL). When approaches to CBME rely on residents to initiate WBA, learners experience tension between seeking WBA for learning and for establishing competence. How learners resolve this tension may lead to unintended consequences for both AfL and AoL. We sought to explore the factors that impact both decisions to seek and not to seek WBA and use the findings to build a model of assessment-seeking strategy used by residents. In building this model we consider how the link between WBA and promotion or progression within a program impacts an individual's assessment-seeking strategy. Approach: We conducted 20 semi-structured interviews with internal medicine residents at Queen's University about the factors that influence their decision to seek or avoid WBA. Using grounded theory methodology, we applied a constant comparative analysis to collect data iteratively and identify themes. A conceptual model was developed to describe the interaction of factors impacting the decision to seek and initiate WBA. Findings: Participants identified two main motivations when deciding to seek assessments: the need to fulfill program requirements and the desire to receive feedback for learning. Analysis suggested that these motivations are often at odds with each other. Participants also described several moderating factors that impact the decision to initiate assessments, irrespective of the primary underlying motivation. These included resident performance, assessor factors, training program expectations, and clinical context. A conceptual framework was developed to describe the factors that lead to strategic assessment-seeking behaviors. Insights: Faced with the dual purpose of WBA in CBME, resident behavior in initiating assessment is guided by specific assessment-seeking strategies. Strategies reflect individual underlying motivations, influenced by four moderating factors. These findings have broad implications for programmatic assessment in a CBME context including validity considerations for assessment data used in summative decision-making including readiness for unsupervised practice.

Promoting physician-as-manager identities: The role of the workplace learning environment

Blissett and O'Sullivan contend that workplace learning environment contributes to learners devaluing administrative tasks and, as such, must be considered in strategies to promote a physician‐as‐manager identity.

Test-reduced teaching for stimulation of intrinsic motivation (TRUST): a randomized controlled intervention study

BACKGROUND

The anatomy dissection course is a major part of the first two years of the traditional medical curriculum in Germany. The vast amount of content to be learned and the repeated examination is unanimously perceived by students and teachers as a major stress factor that contributes to the increase of psychosocial stress during the first two years of the course of study. Published interventions for specific stress reduction are scarce.

METHODS

In a randomized, controlled design two intervention groups were compared with a control group (CG) over the whole dissection course (nine measuring points before, during and after first and second semester). The 'Stress Management intervention (IVSM)' targeted at the setting of personal standards, the 'Friendly Feedback intervention (IVFF)' at the context of frequent testing. Quantitative surveys were distributed at nine measuring points. The questionnaire comprised validated instruments and self-developed items regarding stress, positive and negative affect, anxiety, intrinsic and extrinsic motivation, self-efficacy, and perceived performance.

RESULTS

Out of 195 students inscribed in the dissection course, 166 (85%) agreed to participate in the study. The experience of stress during the dissection course was significantly higher in the CG than in the IVFF. Anxiety and negative affect were lower in students of the IVFF while positive affect, intrinsic motivation, and self-efficacy were higher than in the CG. For anxiety and negative affect in the IVSM this was especially seen at the end of the second semester. The self-perceived increase in both knowledge and preparedness for the first big oral and written examination did not differ between the study groups. About three quarters of the participants would choose the intervention 'Friendly Feedback' if given the choice.

CONCLUSIONS

Replacing formal tests with friendly feedback has proven to be an effective measure to reduce stress and negative affect and foster positive affect, self-efficacy, and intrinsic motivation, while it did not impair self-perceived academic performance.

The Intersection of Growth Mindset and Accreditation in Pharmacy Education

OBJECTIVES

To describe existing growth mindset literature within pharmacy and health care education, describe how a growth mindset can be beneficial in the accreditation process, and propose potential ways to promote a growth mindset in faculty, preceptors, students, and staff within pharmacy education.

FINDINGS

To help pharmacy learners develop a growth mindset, existing literature emphasizes the need for a shift toward and aligning assessment with a growth mindset, helping to create self-directed adaptive learners, leading to health care providers who can adjust their practice to tackle expected and unexpected challenges throughout their careers. Strategies to create a culture of growth mindset identified include training faculty and learners on growth mindset and developing new assessments that track a learner's growth. Recommendations for pharmacy educators include encouraging educators to assess their own growth mindset and use a variety of teaching methods and provide feedback on learner effort that encourages the process of learning rather than focusing on individual attributes, traits, and results.

SUMMARY

Growth mindset intersects with accreditation standards for both professional degree programs and providers of continuing pharmacy education. Continuing professional development process is one way to encourage faculty, staff, and students to develop a growth mindset. While a growth mindset can have many positive impacts on pharmacy accreditation, it is essential to recognize that achieving and maintaining accreditation is a multifaceted process involving numerous factors. A growth mindset can positively influence pharmacy education accreditation by fostering a culture of continuous improvement, innovation, resilience, student-centeredness, data-driven decision-making, collaboration, and effective leadership.

Prediction of general practice licensure/certification examination performance by a measure of 'questionable' activities in clinical practice: a retrospective cohort study

PURPOSE

'Low-value' clinical care and medical services are 'questionable' activities, being more likely to cause harm than good or with disproportionately low benefit relative to cost. This study examined the predictive ability of the QUestionable In Training Clinical Activities Index (QUIT-CAI) for general practice (GP) registrars' (trainees') performance in Australian GP Fellowship examinations (licensure/certification examinations for independent GP).

METHODS

The study was nested in ReCEnT, an ongoing cohort study in which Australian GP registrars document their in-consultation clinical practice. Outcome factors in analyses were individual registrars' scores on the three Fellowship examinations ('AKT', 'KFP', and 'OSCE' examinations) and pass/fail rates during 2012-21. Analyses used univariable and multivariable regression (linear or logistic, as appropriate). The study factor in each analysis was 'QUIT-CAI score percentage'-the percentage of times a registrar performed a QUIT-CAI clinical activity when 'at risk' (i.e. when managing a problem where performing a QUIT-CAI activity was a plausible option).

RESULTS

A total of 1265, 1145, and 553 registrars sat Applied Knowledge Test, Key Features Problem, and Objective Structured Clinical Exam examinations, respectively. On multivariable analysis, higher QUIT-CAI score percentages (more questionable activities) were significantly associated with poorer Applied Knowledge Test scores (P = .001), poorer Key Features Problem scores (P = .003), and poorer Objective Structured Clinical Exam scores (P = .005). QUIT-CAI score percentages predicted Royal Australian College of General Practitioner exam failure [odds ratio 1.06 (95% CI 1.00, 1.12) per 1% increase in QUIT-CAI, P = .043].

CONCLUSION

Performing questionable clinical activities predicted poorer performance in the summative Fellowship examinations, thereby validating these examinations as measures of actual clinical performance (by our measure of clinical performance, which is relevant for a licensure/certification examination).

Pediatric Critical Care Fellow Perception of Learning through Virtual Reality Bronchoscopy

Background

Virtual reality (VR) simulators have revolutionized training in bronchoscopy, offering unrestricted availability in a low-stakes learning environment and frequent assessments represented by automatic scoring. The VR assessments can be used to monitor and support learners' progression. How trainees perceive these assessments needs to be clarified.

Objective

The objective of this study was to examine what assessments learners select to document and receive feedback on and what influences their decisions.

Methods

We used a sequential explanatory mixed methods strategy. All participants were pediatric critical care medicine trainees requiring competency in bronchoscopy skills. During independent simulation practice, we collected the number of learning-focused practice attempts (scores not recorded), assessment-focused practice (scores recorded and reviewed by the instructor for feedback), and the amount of time each attempt lasted. After simulation training, we conducted interviews to explore learners' perceptions of assessment.

Results

There was no significant difference in the number of attempts for each practice type. The average time per learning-focused attempt was almost three times longer than the assessment-focused attempt (mean [standard deviation] 16 ± 1 min vs. 6 ± 3 min, respectively; P < 0.05). Learners perceived documentation of their scores as high stakes and only recorded their better scores. Learners felt safer experimenting if their assessments were not recorded.

Conclusion

During independent practice, learners took advantage of automatic assessments generated by the VR simulator to monitor their progression. However, the recording of scores from the simulation program to document learners' trajectory to a set goal was perceived as high stakes, discouraging learners from seeking supervisor feedback.

Thresholds of becoming: an exploration of threshold concepts as a means to support professional identity formation in competency-based curricula

Inherent in every clinical preceptor's role is the ability to understand the learning needs of individual trainees, enabling them to meet their potential. Competency-based medical education frameworks have been developed to this end, but efforts to identify behaviours and activities that define competence are based on mapping knowledge, skills and ability, which can be difficult to integrate into a comprehensive picture of who the trainee is becoming. Professional identity formation, in contrast, prioritizes attention to who trainees are becoming, but provision of detailed guidance to preceptors on how to best support this form of development is challenging. The tension that results limits our ability to optimally support learners as strengths in competency development may mask professional identity development gaps and vice versa. To address this tension, this paper examines how the theory of threshold concepts - troublesome ideas that, once appreciated, fundamentally change how you understand and approach a particular activity - can shine light on professional identity formation and its relationship with developing competence. The recognition and identification of threshold concepts is offered as a means to improve our ability to identify, discuss and support behaviours and actions that impact the learner's capacity to act competently as they develop their identity at various stages of training.

Efficacy of the Gulu University Reproductive Health Simulation Training for final year medical students and interns: a before-and-after study

Background

Reproductive health emergencies, such as postpartum hemorrhage, contribute significantly to maternal and neonatal morbidity and mortality in Uganda due to knowledge and skills gaps. Medical interns, intern midwives, and nurses are crucial as frontline healthcare workers in responding to these emergencies. Our proposed hands-on strategy involves comprehensive simulation-based training (SBT) to equip these healthcare workers with the essential knowledge to manage common reproductive health emergencies and procedures in the country.

Objectives

The study aimed to assess the effectiveness of comprehensive SBT in improving the knowledge of interns and fifth-year medical students on reproductive health emergencies and procedures at Gulu University and its Teaching Hospitals in Uganda.

Design

A before-and-after study.

Methods

A 4-day SBT was conducted for fifth-year medical students and interns (nurses, midwives, and doctors) at Gulu University Teaching Hospitals, focusing on reproductive health emergencies. Pre- and post-tests with 40 multiple-choice questions were used to evaluate knowledge enhancement, the scores were summarized as medians and interquartile ranges. Paired sample t-tests was used to test the difference in pre- and post-test scores. Independent sample t-tests compared median post-test results between interns and students, with a p-value <0.05 considered significant.

Results

A total of 153 participants were enrolled, the majority being males (78.4%, n = 120) and medical students (73.9%, n = 113). Among the 40 interns, 55% (n = 22) were doctors, 30% (n = 12) were midwives, and 15% (n = 6) were nurses. The study participants showed an increase in knowledge, with median post-test scores higher than pre-test scores for all participants [63% (interquartile ranges, IQR: 57-71%) versus 49% (42-54%), with a median difference of 14% (8-23%), p < 0.001].

Conclusion

The SBT effectively imparts key knowledge competencies to the interns and fifth-year medical students. We recommend that SBT be included as part of the course units that students should take and for continuous medical education for qualified healthcare workers in resource-limited settings.

Modification of Fundamentals of Laparoscopic Surgery Manual Tasks Simulation Curriculum With the Addition of the Vaginal Cuff Closure Training

OBJECTIVE

Fundamentals of Laparoscopic Surgery (FLS) is a multiple-choice test and a manual skills exam using simulation that Obstetrics and Gynecology (OBGYN) residents must pass to qualify for board certification. There is insufficient validity evidence supporting the use of FLS as a high-stakes exam. This study examines the correlation between OBGYN residents' performance on the FLS manual tasks and simulated vaginal cuff closure.

METHODS

We compared residents' performance on FLS tasks with simulated vaginal cuff suturing on a model. During the first coached simulation session, after completion of training on the standard 5 FLS tasks, residents were coached on vaginal cuff closure using a simulated model placed inside the standard FLS box trainer. At a subsequent session, their performance was scored using the Global Operative Assessment of Laparoscopic Skills Scale (GOALS) and a second task-specific metric, and these scores were compared to their official FLS score.

RESULTS

Twenty-nine residents completed the vaginal cuff simulation training between June 2019 and November 2021. Nineteen of the 29 were able to complete the cuff closure with the mean time to completion being 14.5 minutes. We found no correlation between official manual skills FLS scores and vaginal cuff GOALS scores (rho = -0.02, p = 0.90) or cuff closure assessment tool score (rho = -0.015, p = 0.048). There was also no correlation between time to completion for any FLS task and vaginal cuff closure OSAT scores. All residents reported that they found the cuff to be a useful addition to the FLS curriculum.

CONCLUSIONS

Our study demonstrated that trainee performance on a simulated vaginal cuff closure model did not correlate with official FLS manual tasks skills. This finding adds to the body of evidence disputing the use of FLS as a high-stakes exam to assess laparoscopic skills in gynecology in the relationship with other variables category.

Key aspects of teaching that affect perceived preparedness of medical students for transition to work: insights from the COVID-19 pandemic

BACKGROUND

The self-perceived preparedness of medical students to transition into practising junior doctors has implications for patient safety, graduate well-being and development of professional identity.

AIMS

To examine the impact of changes to final-year education and placements and determine key elements that contribute to self-perceived preparedness for transition to work.

METHODS

An online survey among final-year medical students at one Australian medical school in 2020 (the cohort most affected by the coronarvirus disease 2019 [COVID-19] pandemic), exploring overall self-perceived preparedness and specific competencies, including questions in previous Australian Medical Council/Medical Board of Australia annual national surveys. Quantitative and qualitative content analyses were performed.

RESULTS

Thirty-three percent of eligible participants completed the survey. There was a significant decline in overall self-perceived preparedness among participants (mean preparedness, 3.55 ± 0.88) compared with 2019 graduates from the same medical school (mean preparedness, 4.28 ± 0.64, P < 0.001) and the national average (mean preparedness, 3.81 ± 0.93, P = 0.04). There was a decline in self-perceived preparedness for all specific competencies, with complex competencies more greatly affected. Qualitative content analysis of free text responses identified limitations of an online compared with a face-to-face formal education program and specific aspects of placements, which contribute to perceived preparedness.

CONCLUSION

The current study highlights key aspects of clinical placements and formal teaching programs that contribute to perceptions of preparedness for transition to clinical practice. Relevant experiential learning in the clinical setting, opportunities for deliberate practice of necessary skills (in simulation and the clinical setting) and reflective opportunities from formal teaching programs contribute to perceived preparedness and are important elements to be included in the final phase of any medical program.

Grappling with key questions about assessment of the Health Advocate role

Introduction

Although the CanMEDS framework sets the standard for Canadian training, health advocacy competence does not appear to factor heavily into high stakes assessment decisions. Without forces motivating uptake, there is little movement by educational programs to integrate robust advocacy teaching and assessment practices. However, by adopting CanMEDS, the Canadian medical education community endorses that advocacy is required for competent medical practice. It's time to back up that endorsement with meaningful action. Our purpose was to aid this work by answering the key questions that continue to challenge training for this intrinsic physician role.

Methods

We used a critical review methodology to both examine literature relevant to the complexities impeding robust advocacy assessment, and develop recommendations. Our review moved iteratively through five phases: focusing the question, searching the literature, appraising and selecting sources, and analyzing results.

Results

Improving advocacy training relies, in part, on the medical education community developing a shared vision of the Health Advocate (HA) role, designing, implementing, and integrating developmentally appropriate curricula, and considering ethical implications of assessing a role that may be risky to enact.

Conclusion

Changes to assessment could be a key driver of curricular change for the HA role, provided implementation timelines and resources are sufficient to make necessary changes meaningful. To truly be meaningful, however, advocacy first needs to be perceived as valuable. Our recommendations are intended as a roadmap for transforming advocacy from a theoretical and aspirational value into one viewed as having both practical relevance and consequential implications.

A survey of clinical empathy training at UK medical schools

BACKGROUND

The benefits of enhancing practitioner empathy include better patient outcomes and improved job satisfaction for practitioners. Evidence suggests empathy can be taught and empathy is listed as an outcome for graduates in the General Medical Council requirements. Despite this, empathy training is not mandatory on medical school curricula and the extent to which medical students are given empathy-specific training is unknown.

AIM

To conduct a survey of empathy training currently offered to medical students in UK medical schools.

METHODS

An invitation to participate in an online survey was sent to all UK medical schools (n = 40). The survey was developed through a consultancy and pilot process to ensure validity and reliability. Questions explored what empathy-focused training is offered, and asked educators whether or not they believed that current provision of empathy training is sufficient. In parallel, medical school websites were searched to identify what information regarding empathy-focused training is described as being part of the degree course. Descriptive statistics were used to describe empathy training delivery from the results of the online materials survey and closed survey questions. Thematic analysis was used to explore free text comments.

RESULTS

Response rate was 70% (28/40), with 28 medical schools included in the analysis. Twenty-six schools reported that their undergraduate curriculum included some form of empathy-focused training with variation in what, when and how this is delivered. Thematic analysis revealed two overarching themes with associated sub-themes: (i) empathy-focused training and development (considering where, when and how empathy training should be integrated); (ii) challenges presented by including empathy on the curriculum (considering the system, students and faculty). All schools agreed empathy training should be on the undergraduate curriculum.

CONCLUSION

This is the first nationwide survey of empathy-focused training at UK medical schools. While some form of empathy-focused training appears to be provided on the undergraduate curriculum at most UK medical schools, empathy is rarely specifically assessed. Most medical educators do not feel their school does enough to promote empathy and the majority would like to offer more.

Teaching perspectives on the communication of difficult news of genetic conditions to medical students

Informing parents that their child has a diagnosis of Down syndrome (DS) is a common example of the delivery of unexpected or difficult news. Expectations and life planning will change, and if detected prenatally, discussions might include the option of pregnancy termination. Medical school curricula currently include training in breaking unexpected news; however, it is difficult to teach and assess. We use the perspectives of clinicians, educators, and a medical student who is the parent of a child with DS to frame a discussion on teaching, practicing, and assessing communication of difficult news in human genetics during medical school.

Interprofessional education focused on medication safety: a systematic review

Safe medication use necessitates interprofessional working, with calls to enhance interprofessional education (IPE) focusing on medication safety (MS) in healthcare professional (HCP) curricula. Little is known about the design, delivery or evaluation of such activities. This systematic literature review describes MS-focused IPE activities in pre-qualification HCP programmes. MedLine, EMBASE, CINAHL and ERIC were searched, relevant studies identified and data extracted. The McGill Mixed Methods Appraisal Tool was employed. The 3P (presage-process-product) theory structured deductive analysis. Thirty-one studies were included, reporting on 30 activities, mostly undertaken in North America or United Kingdom. Presage/Design: Most reported activities involved pharmacy, nursing, medical or physician assistant students learning with one or more other HCP group. Few studies matched student groups' skills or experiences. Few studies reported theoretical underpinnings. Process/Delivery: Multiple pedagogical approaches were employed, mostly social construction, and low- and high-fidelity simulation-based learning. Few studies reported learning outcomes or summative assessment, more reported formative assessment. Product/evaluation: Outcomes measured were learners' opinions, satisfaction or attitudes toward interprofessional working and findings were generally positive. Few studies reported on student development or outcomes specific to medication safety. Lack of integration of qualitative/quantitative components of mixed methods studies and limited outcome measurements' validity or reliability weakened study quality. MS-focused IPE for pre-qualification HCPs is well received by students. Design of future activities could be enhanced by employing theory and ensuring matching of students' and groups' skills, professional identity and learner attributes to enhance learning in an interprofessional setting. Future delivery should embed MS-focused IPE into the standard curricula to optimize constructive alignment, learner engagement, quality and drive development. The required skillset in pre-qualification HCP programmes to facilitate future safe medication practice, together with the associated learning outcomes and assessment approaches, should be defined. The quality of scholarly studies examining these activities needs improvement.

Informed self-assessment during the transition to medical school: a longitudinal narrative study

OBJECTIVES

To explore how medical students' narratives of informed self-assessment (ISA) change during their first 18 months of study.

DESIGN

This longitudinal study used student narratives drawn from qualitative interviews and written reflections during the transition to medical school, to examine changes in ISA. Our analysis was informed by Situated Cognition Theory which recognises the impact and interplay of personal and environmental factors in cognition.

SETTING

To study medicine, first year students need to adapt their self-regulated learning in the context of a new peer group, study demands and educational culture. During this adaptation, students need to seek and interpret available cues to inform their self-assessment.

PARTICIPANTS

Longitudinal data were collected at five time points over 18 months from a diverse sample of seven first year medical students in an undergraduate medical programme, including 13.5 hours of interviews and 12 written reflections.

RESULTS

Before and after starting medical school, the participants' self-assessments were informed by environmental influences (exam results and comparison with peers), and personal influences (fear of failure and anxiety about not belonging). Early uncertainty meant self-assessments were overestimated and underestimated.By the end of first year, an enhanced sense of belonging coincided with less fear of failure, less emphasis on exam performance and reduced competition with peers. Self-assessments became increasingly informed by evidence of clinical skills and knowledge gained related to future professional competence.

CONCLUSION

Influences on medical students' ISAs change during the transition to studying medicine. A greater sense of belonging, and evidence of progress towards clinical competence became more important to self-assessment than comparison with peers and exam performance. Our findings reinforce the importance of formative assessments, opportunities to study and socialise with peers and early clinical experiences during first year. These experiences enhance ISA skills during the transition to medical school.

Putting nutrition education on the table: development of a curriculum to meet future doctors' needs

COVID-19 has further exacerbated trends of widening health inequalities in the UK. Shockingly, the number of years of life lived in general good health differs by over 18 years between the most and least deprived areas of England. Poor diets and obesity are established major risk factors for chronic cardiometabolic diseases and cancer, as well as severe COVID-19. For doctors to provide the best care to their patients, there is an urgent need to improve nutrition education in undergraduate medical school training. With this imperative, the Association for Nutrition established the Inter-Professional Working Group on Medical Education (AfN IPG) to develop a new, modern undergraduate nutrition curriculum for medical doctors. The AfN IPG brought together expertise from nutrition, dietetic and medical professionals, representing the National Health Service, royal colleges, medical schools and universities, government public health departments, learned societies, medical students and nutrition educators. The curriculum was developed with the key objective of being implementable through integration with the current undergraduate training of medical doctors. Through an iterative and transparent consultative process, 13 key nutritional competencies, to be achieved through mastery of 11 graduation fundamentals, were established. The curriculum to facilitate the achievement of these key competencies is divided into eight topic areas, each underpinned by a learning objective statement and teaching points detailing the knowledge and skills development required. The teaching points can be achieved through clinical teaching and a combination of facilitated learning activities and practical skills acquisition. Therefore, the nutrition curriculum enables mastery of these nutritional competencies in a way that will complement and strengthen medical students' achievement of the General Medical Council Outcomes for Graduates. As nutrition is an integrative science, the AfN IPG recommends the curriculum is incorporated into initial undergraduate medical studies before specialist training. This will enable our future doctors to recognise how nutrition is related to multiple aspects of their training, from physiological systems to patient-centred care, and acquire a broad, inclusive understanding of health and disease. In addition, it will facilitate medical schools to embed nutrition learning opportunities within the core medical training, without the need to add in a large number of new components to an already crowded programme or with additional burden to teaching staff. The undergraduate nutrition curriculum for medical doctors is designed to support medical schools to create future doctors who will understand and recognise the role of nutrition in health. Moreover, it will equip front-line staff to feel empowered to raise nutrition-related issues with their patients as a fundamental part of enhanced care and to appropriately refer on for nutrition support with a registered nutritionist (RNutr)/registered associate nutritionist (ANutr) or a registered dietitian (RD) where this is likely to be beneficial.

Assessment for Learning with Ungraded and Graded Assessments

Introduction

Assessment for learning has many benefits, but learners will still encounter high-stakes decisions about their performance throughout training. It is unknown if assessment for learning can be promoted with a combination model where scores from some assessments are factored into course grades and scores from other assessments are not used for course grading.

Methods

At the University of Utah School of Medicine, year 1-2 medical students (MS) completed multiple-choice question quiz assessments and final examinations in six systems-based science courses. Quiz and final examination performance counted toward course grades for MS2017-MS2018. Starting with the MS2020 cohort, quizzes no longer counted toward course grades. Quiz, final examination, and Step 1 scores were compared between ungraded quiz and graded quiz cohorts with independent samples t-tests. Student and faculty feedback was collected.

Results

Quiz performance was not different for the ungraded and graded cohorts (p = 0.173). Ungraded cohorts scored 4% higher on final examinations than graded cohorts (p ≤ 0.001, d = 0.88). Ungraded cohorts scored above the national average and 11 points higher on Step 1 compared to graded cohorts, who had scored below the national average (p ≤ 0.001, d = 0.64). During the study period, Step 1 scores increased by 2 points nationally. Student feedback was positive, and faculty felt it improved their relationship with students.

Discussion

The change to ungraded quizzes did not negatively affect final examination or Step 1 performance, suggesting a combination of ungraded and graded assessments can effectively promote assessment for learning.

Putting nutrition education on the table: development of a curriculum to meet future doctors' needs

COVID-19 has further exacerbated trends of widening health inequalities in the UK. Shockingly, the number of years of life lived in general good health differs by over 18 years between the most and least deprived areas of England. Poor diets and obesity are established major risk factors for chronic cardiometabolic diseases and cancer, as well as severe COVID-19. For doctors to provide the best care to their patients, there is an urgent need to improve nutrition education in undergraduate medical school training.With this imperative, the Association for Nutrition established an Interprofessional Working Group on Medical Education (AfN IPG) to develop a new, modern undergraduate nutrition curriculum for medical doctors. The AfN IPG brought together expertise from nutrition, dietetic and medical professionals, representing the National Health Service (NHS), royal colleges, medical schools and universities, government public health departments, learned societies, medical students, and nutrition educators. The curriculum was developed with the key objective of being implementable through integration with the current undergraduate training of medical doctors.Through an iterative and transparent consultative process, thirteen key nutritional competencies, to be achieved through mastery of eleven graduation fundamentals, were established. The curriculum to facilitate the achievement of these key competencies is divided into eight topic areas, each underpinned by a learning objective statement and teaching points detailing the knowledge and skills development required. The teaching points can be achieved through clinical teaching and a combination of facilitated learning activities and practical skill acquisition. Therefore, the nutrition curriculum enables mastery of these nutritional competencies in a way that will complement and strengthen medical students' achievement of the General Medical Council (GMC) Outcome for Graduates.As nutrition is an integrative science, the AfN IPG recommends that the curriculum is incorporated into initial undergraduate medical studies before specialist training. This will enable our future doctors to recognise how nutrition is related to multiple aspects of their training, from physiological systems to patient-centred care, and acquire a broad, inclusive understanding of health and disease. In addition, it will facilitate medical schools to embed nutrition learning opportunities within the core medical training, without the need to add in a large number of new components to an already crowded programme or with additional burden for teaching staff.The undergraduate nutrition curriculum for medical doctors is designed to support medical schools to create future doctors who will understand and recognise the role of nutrition in health. Moreover, it will equip frontline staff to feel empowered to raise nutrition-related issues with their patients as a fundamental part of enhanced care and to appropriately refer on for nutrition support with a registered associate nutritionist/registered nutritionist (ANutr/RNutr) or registered dietitian (RD) where this is likely to be beneficial.

Cancel culture: exploring the unintended consequences of cancelling the Canadian national licensing clinical examination

Assessment drives learning. However, when it comes to high-stakes examinations (e.g., for licensure or certification), these assessments of learning may be seen as unnecessary hurdles by some. Licensing clinical skills assessment in particular have come under fire over the years. Recently, assessments such as the Medical Council of Canada Qualifying Examination Part II, a clinical skills objective structured clinical examination, have been permanently cancelled. The authors explore potential consequences of this cancellation including those that are inadvertent and undesirable. Future next steps for clinical skills assessment are explored.

Novel use of an OSCE to assess medical students' responses to a request for a low value diagnostic imaging test: A mixed methods analysis

OBJECTIVE

Evaluate medical students' communication skills with a standardized patient (SP) requesting a low value test and describe challenges students identify in addressing the request.

METHODS

In this mixed-methods study, third-year students from two medical schools obtained a history, performed a physical examination, and counseled an SP presenting with uncomplicated low back pain who requests an MRI which is not indicated. SP raters evaluated student communication skills using a 14-item checklist. Post-encounter, students reported whether they ordered an MRI and challenges faced.

RESULTS

Students who discussed practice guidelines and risks of unnecessary testing with the SP were less likely to order an MRI. Students cited several challenges in responding to the SP request including patient characteristics and circumstances, lack of knowledge about MRI indications and alternatives, and lack of communication skills to address the patient request.

CONCLUSIONS

Most students did not order an MRI for uncomplicated LBP, but only a small number of students educated the patient about the evidence to avoid unnecessary imaging or the harm of unnecessary testing.

PRACTICE IMPLICATIONS

Knowledge about unnecessary imaging in uncomplicated LBP may be insufficient to adhere to best practices and longitudinal training in challenging conversations is needed.

Profiles for Success: Examining the Relationship between Student Profiles and Clerkship Performance Using Latent Profile Analysis

PHENOMENON

Learning is a complex phenomenon in which learners can vary in their learning orientation, learning approaches, and perception of the learning environment. Learners may be characterized as mastery oriented or performance oriented, and this learning orientation can influence their learning approaches, such as whether to seek feedback or ask for help. The learning environment includes institutional assessment policies and informal interactions that emphasize outcomes and differentiation among students. When learners perceive the learning environment as performance oriented, they feel competition from other learners and fear negative feedback. This complex interplay of learning orientation, its influence on feedback-seeking behavior, and the environmental influence and reinforcement of both can be captured through learner profiles.

APPROACH

In this multi-institutional cross-sectional survey study of students from six U.S. medical schools, we investigated learner profiles that characterize how these domains manifest together across individual learners. We then determined how these profiles are associated with students' clerkship grades. Measures included the Patterns of Adaptive Learning for learning orientation, self-reports for learning approaches, and perceptions of the learning environment. We used latent profile analysis to cluster students who share common characteristics around orientation, behavior, and environment. The relationship between these profiles and the percentage of honors earned was examined using multiple regression analysis.

FINDINGS

We found four distinct learner profiles within 666 of 974 (68%) analyzable responses: (1) performance oriented with negative perceptions of environment, (2) mastery oriented with desirable learning approaches, (3) "average" group, and (4) mastery orientation only. Profile 1 (M = 39%) and Profile 4 (M = 38%) students received fewer clerkship honors compared with other profiles. Profile 2 students earned a significantly higher percentage of honors grades (M = 54%) compared with other students. Profile 3 students (average group) earned the second highest percentage of honors (M = 46%). Effect sizes comparing the percentage of honors varied from 0.32 to 0.65, indicating medium to large differences.

INSIGHTS

The highest performers, as measured by clerkship honors grades, were mastery oriented, perceived a more positive learning environment, and were comfortable asking questions and seeking feedback for their learning. The lowest performing students were performance oriented and had negative perceptions of their learning environments. Contrary to previous findings, we found that the relationship between mastery orientation and learning approaches is not unidirectional but rather seems to be mediated by perceptions of the learning environment.

Patient-present teaching in the clinic: Effect on agency and professional behaviour

BACKGROUND/PURPOSE

Although much has been written about the medical learning environment, the patient, who is the focus of care, is rarely the focus in this literature. The purpose of this study was to explore the role of the patient as an active participant with agency in the medical learning environment from the standpoint of the learner, the attending physician, and most importantly, the patient. We hoped to gain insights into the mechanisms that can reinforce professional values such as patient-centred and respectful behaviours in a patient-present learning environment.

METHODS

We conducted this study in an ambulatory internal medicine clinic using 'patient-present' clinic visits. All case presentations occurred in examination rooms with the patient. We invited participants (attending physicians, undergraduate and postgraduate learners, patients and family members) to participate in semistructured interviews after each clinic visit to explore the impact of the patient-present learning environment. We recruited 34 participants in the study; 10 attending physicians, 12 learners, 10 patients and 2 family members. We analysed the data deductively using a conceptual framework of agency.

SUMMARY/RESULTS

We identified three major insights: (1) Patients felt engaged and valued opportunities to be heard; (2) Attending physicians and learners reported a more respectful learning environment and a positive though challenging teaching and learning experience; and (3) A hidden curriculum emerged in a performance-based view of professional behaviour.

CONCLUSIONS

Patient-present teaching engaged patients and enhanced their agency by recasting the patient as the central focus within the healthcare encounter. We identified a tension between performing and learning. This study adds new insights to the concept of patient centredness and professionalism from the perspectives of all participants in the medical teaching and learning environment.

Observed structured clinical examination as a means of assessing clinical skills competencies of ANPs

Observed structured clinical examinations (OSCEs) are a common method of assessment within higher education to prepare for the advanced nurse practitioner (ANP) role. This article reviews a wide range of literature relating to OSCE assessment in the healthcare arena, from an ANP, interprofessional and advanced clinical practice perspective. Theories underpinning OSCE and advanced nursing roles are explored, with relevant supporting literature indicating how established OSCEs can become integrated with other methods to improve outcomes for this level of practice. Alternative assessments are explored with specific reference to the context of the education delivered, and the suitability for higher education today.

Feedback and coaching

If used thoughtfully and with intent, feedback and coaching will promote learning and growth as well as personal and professional development in our learners. Feedback is an educational tool as well as a social interaction between learner and supervisor, in the context of a respectful and trusting relationship. It challenges the learner's thinking and supports the learner's growth. Coaching is an educational philosophy dedicated to supporting learners' personal and professional development and growth and supporting them to reach their potential. In clinical education, feedback is most effective when it is explicitly distinguished from summative assessment. Importantly, feedback should be about firsthand observed behaviors (which can be direct or indirect) and not about information which comes from a third party. Learners are more receptive to feedback if it comes from a source that they perceive as credible, and with whom they have developed rapport. The coaching relationship between learner and supervisor should also be built on mutual trust and respect. Coaching can be provided in the moment (feedback on everyday clinical activities that leads to performance improvement, even with short interaction with a supervisor) and over time (a longer term relationship with a supervisor in which there is reflection on the learner's development and co-creation of new learning goals). Feedback and coaching are most valuable when the learner and teacher exhibit a growth mindset. At the organizational level, it is important that both the structures and training are in place to ensure a culture of effective feedback and coaching in the clinical workplace.Conclusions: Having a thoughtful and intentional approach to feedback and coaching with learners, as well as applying evidence-based principles, will not only contribute in a significant way to their developmental progression, but will also provide them with the tools they need to have the best chance of achieving competence throughout their training. What is Known: • Feedback and coaching are key to advancing the developmental progression of trainees as they work towards achieving competence. • Feedback is not a one-way delivery of specific information from supervisor to trainee, but rather a social interaction between two individuals in which trust and respect play a key role. • Provision of effective feedback may be hampered by confusing formative (supporting trainee learning and development) and summative (the judgment that is made about a trainee's level of competence) purposes. What is New: • Approaches to both the provision of feedback/coaching and the assessment of competence must be developed in parallel to ensure success in clinical training programs. • Faculty development is essential to provide clinical teachers with the skills to provide effective feedback and coaching. • Coaching's effectiveness relies on nurturing strong trainee-supervisor relationships, ensuring high-quality feedback, nourishing a growth mindset, and encouraging an institutional culture that embraces feedback and coaching.

Emotion in remediation: A scoping review of the medical education literature

OBJECTIVES

Remediation can be crucial and high stakes for medical learners, and experts agree it is often not optimally conducted. Research from other fields indicates that explicit incorporation of emotion improves education because of emotion's documented impacts on learning. Because this could present an important opportunity for improving remediation, we aimed to investigate how the literature on remediation interventions in medical education discusses emotion.

METHODS

The authors used Arksey and O'Malley's framework to conduct a scoping literature review of records describing remediation interventions in medical education, using PubMed, CINAHL Complete, ERIC, Web of Science and APA PsycInfo databases, including all English-language publications through 1 May 2020 meeting search criteria. They included publications discussing remediation interventions either empirically or theoretically, pertaining to physicians or physician trainees of any level. Two independent reviewers used a standardised data extraction form to report descriptive information; they reviewed included records for the presence of mentions of emotion, described the mentions and analysed results thematically.

RESULTS

Of 1644 records, 199 met inclusion criteria and were reviewed in full. Of those, 112 (56%) mentioned emotion in some way; others focused solely on cognitive aspects of remediation. The mentions of emotion fell into three themes based on when the emotion was cited as present: during regular coursework or practice, upon referral for remediation and during remediation. One-quarter of records (50) indicated potential intentional incorporation of emotion into remediation programme design, but they were non-specific as to how emotions related to the learning process itself.

CONCLUSION

Even though emotion is omnipresent in remediation, medical educators frequently do not factor emotion into the design of remediation approaches and rarely explicitly utilise emotion to improve the learning process. Applications from other fields may help medical educators leverage emotion to improve learning in remediation, including strategies to frame and design remediation.

Medical licensing examinations in both Sweden and the US favor pharmacology over lifestyle

Low priority of disease prevention and health promotion in medical education may contribute to lack of lifestyle-counseling in clinical practice. Pharmacology-related knowledge is valued 5 times higher compared to lifestyle-related knowledge in examinations on noncommunicable diseases (NCDs) in undergraduate medical education in Sweden. This study aims to establish (i) whether medical licensing examinations are biased to favor pharmacology- over lifestyle-related knowledge and (ii) whether such a bias is present in both Sweden and the US. We identified 204 NCD-related questions from previous Swedish licensing examinations, and 77 cases from a U.S. question bank commonly used to prepare for the United States Medical Licensing Examination® (USMLE®) Step 3. With the help of expected correct answers, we determined distribution of points attainable for knowledge in the respective category (lifestyle / pharmacology / other) for 5 major NCDs: coronary heart disease (CHD), chronic obstructive pulmonary disease (COPD), diabetes, hypertension, and stroke. The percentage of points attainable for lifestyle-related knowledge was 6.7 (95% CI 4.1-9.3) in Sweden and 4.6 (95%CI 0.0-9.1) in the U.S. The respective percentages for pharmacology-related knowledge were 32.6 (95% CI 26.3-38.8) and 44.5 (95% CI 33.2-55.8) percent. The pharmacology vs. lifestyle-quotas were 4.9 in Sweden and 9.8 in the U.S. Likelihoods of equal emphasis on lifestyle and pharmacology in NCDs was < 0.001 in both countries. There is a marked preference for pharmacology over lifestyle in medical licensing examinations in both Sweden and the U.S. Newly qualified doctors may be inadequately prepared to address preventable causes of NCDs.

Assessment to Optimize Learning Strategies: A Qualitative Study of Student and Faculty Perceptions

Phenomenon: The format of medical knowledge assessment can promote students' use of effective learning strategies from the learning sciences literature, such as elaboration, interleaving, retrieval practice, and distributed learning. Assessment format can also influence faculty teaching. Accordingly, our institution implemented a new assessment strategy in which pre-clerkship medical students answered weekly formative quizzes with constructed response questions (also referred to as open-ended questions) and multiple-choice questions in preparation for summative open-ended question examinations, to support students' use of recommended learning strategies. Our qualitative study explored medical student and faculty perceptions of this assessment strategy on learning and teaching. Approach: We conducted semi-structured interviews with 16 second-year medical students to explore their preparation for quizzes and summative examinations. We also interviewed 10 faculty responsible for writing and grading these assessments in the pre-clerkship foundational sciences curriculum regarding their approach to writing assessments and rubrics, and their perceptions of how their teaching may have changed with this assessment strategy. We analyzed interview transcripts using thematic analysis with a priori sensitizing concepts from the learning sciences literature. Findings: We identified four major themes characterizing student and faculty perceptions of weekly formative quizzes and summative OEQ examinations. Participants found that this assessment strategy helped (1) prioritize conceptual understanding, (2) simulate clinical problem solving, and (3) engage students and faculty in continuous improvement in their approach to learning or teaching. Faculty and students also recognized challenges and potential tradeoffs associated with these assessment formats. Participants identified (4) facilitators and barriers when implementing this assessment strategy. Insights: Our findings suggested that assessment of medical knowledge through weekly formative quizzes and summative open-ended question examinations can facilitate students' use of effective learning strategies. Faculty also recognized improvements in their teaching and in quality of assessment. This format of assessment also presented some challenges and potential tradeoffs and significant institutional resources were required for implementation.

Perceptions of the impact of annual review of competence progression (ARCP): a mixed methods case study

The annual review of competence progression (ARCP) is a high-stakes assessment which all UK postgraduate trainees undertake to ensure competence progression. Previous evaluations of the effectiveness of the ARCP as an assessment have reported deficiencies in both validity and reliability, however, there has been little focus on the educational impact of the ARCP.We conducted a mixed methods case study involving questionnaire, interviews and a focus group examining the impact of the ARCP on a respiratory higher specialist training programme. Participants included both trainers and trainees.Perceptions of impact were mixed. The ARCP was reported to promote broad curriculum coverage, enable educational planning, provide educational governance and facilitate relationships with supervisors. However, participants reported that activities promoted by the ARCP may detract from learning and that issues of reliability and validity undermined the process. In some cases, this was reported to lead to disillusionment and stress for trainees. Concerns were raised that the process promoted a reductionist approach to education.This research has resulted in several changes to local training, however, it has potential implications for the ARCP as a wider process. Trainers should be cognisant of the shortcomings of assessments and their impact on trainees, training and the future of the profession.

Rethinking Assessment Design: Evidence-Informed Strategies to Boost Educational Impact in the Anatomical Sciences

University assessment is in the midst of transformation. Assessments are no longer designed solely to determine that students can remember and regurgitate lecture content, nor in order to rank students to aid with some future selection process. Instead, assessments are expected to drive, support, and enhance learning and to contribute to student self-assessment and development of skills and attributes for a lifetime of learning. While traditional purposes of certifying achievement and determining readiness to progress remain important, these new expectations for assessment can create tensions in assessment design, selection, and deployment. With the recognition of these tensions, three contemporary approaches to assessment in medical education are described. These approaches include careful consideration of the educational impact of assessment-before, during (test or recall enhanced learning) and after assessments; development of student (and staff) assessment literacy; and planning of cohesive systems of assessment (with a range of assessment tools) designed to assess the various competencies demanded of future graduates. These approaches purposefully straddle the cross purposes of assessment in modern health professions education. The implications of these models are explored within the context of medical education and then linked with contemporary work in the anatomical sciences in order to highlight current synergies and potential future innovations when using evidence-informed strategies to boost the educational impact of assessments.

Simulated Patient Role-Plays with Consumers with Lived Experience of Mental Illness Post-Mental Health First Aid Training: Interrater and Test Re-Test Reliability of an Observed Behavioral Assessment Rubric

Mental Health First Aid (MHFA) training teaches participants how to assist people experiencing mental health problems and crises. Observed behavioral assessments, post-training, are lacking, and the literature largely focuses on self-reported measurement of behaviors and confidence. This study explores the reliability of an observed behavioral assessment rubric used to assess pharmacy students during simulated patient (SP) role-play assessments with mental health consumers. Post-MHFA training, pharmacy students (n = 528) participated in SP role-play assessments (n = 96) of six mental health cases enacted by consumers with lived experience of mental illness. Each assessment was marked by the tutor, participating student, and consumer (three raters). Non-parametric tests were used to compare raters' means scores and pass/fail categories. Interrater reliability analyses were conducted for overall scores, as well as pass/fail categories using intra-class correlation coefficient (ICC) and Fleiss' Kappa, respectively. Test re-test reliability analyses were conducted using Pearson's correlation. For interrater reliability analyses, the intra-class correlation coefficient varied from poor-to-good to moderate-to-excellent for individual cases but was moderate-to-excellent for combined cases (0.70; CI 0.58-0.80). Fleiss' Kappa varied across cases but was fair-to-good for combined cases (0.57, p < 0.001). For test re-test reliability analyses, Pearson's correlation was strong for individual and combined cases (0.87; p < 0.001). Recommended modifications to the rubric, including the addition of barrier items, scoring guides, and specific examples, as well as the creation of new case-specific rubric versions, may improve reliability. The rubric can be used to facilitate the measurement of actual, observed behaviors post-MHFA training in pharmacy and other health care curricula.

Standardization in the MSPE: Key Tensions for Learners, Schools, and Residency Programs

The Medical Student Performance Evaluation (MSPE), which summarizes a medical student's academic and professional undergraduate medical education performance and provides salient information during the residency selection process, faces persistent criticisms regarding heterogeneity and obscurity. Specifically, MSPEs do not always provide the same type or amount of information about students, especially from diverse schools, and important information is not always easy to find or interpret. To address these concerns, a key guiding principle from the Recommendations for Revising the MSPE Task Force of the Association of American Medical Colleges (AAMC) was to achieve "a level of standardization and transparency that facilitates the residency selection process." Benefits of standardizing the MSPE format include clarification of performance benchmarks or metrics, consistency across schools to enhance readability, and improved quality. In medical education, standardization may be an important mechanism to ensure accountability of the system for all learners, including those with varied backgrounds and socioeconomic resources. In this article, members of the aforementioned AAMC MSPE task force explore 5 tensions inherent in the pursuit of standardizing the MSPE: (1) presenting each student's individual characteristics and strengths in a way that is relevant, while also working with a standard format and providing standard content; (2) showcasing school-specific curricular strengths while also demonstrating standard evidence of readiness for internship; (3) defining and achieving the right amount of standardization so that the MSPE provides useful information, adds value to the residency selection process, and is efficient to read and understand; (4) balancing reporting with advocacy; and (5) maintaining standardization over time, especially given the tendency for the MSPE format and content to drift. Ongoing efforts to promote collaboration and trust across the undergraduate to graduate medical education continuum offer promise to reconcile these tensions and promote successful educational outcomes.

Driving lesson or driving test? : A metaphor to help faculty separate feedback from assessment

Although there is consensus in the medical education world that feedback is an important and effective tool to support experiential workplace-based learning, learners tend to avoid the feedback associated with direct observation because they perceive it as a high-stakes evaluation with significant consequences for their future. The perceived dominance of the summative assessment paradigm throughout medical education reduces learners' willingness to seek feedback, and encourages supervisors to mix up feedback with provision of 'objective' grades or pass/fail marks. This eye-opener article argues that the provision and reception of effective feedback by clinical supervisors and their learners is dependent on both parties' awareness of the important distinction between feedback used in coaching towards growth and development (assessment for learning) and reaching a high-stakes judgement on the learner's competence and fitness for practice (assessment of learning). Using driving lessons and the driving test as a metaphor for feedback and assessment helps supervisors and learners to understand this crucial difference and to act upon it. It is the supervisor's responsibility to ensure that supervisor and learner achieve a clear mutual understanding of the purpose of each interaction (i.e. feedback or assessment). To allow supervisors to use the driving lesson-driving test metaphor for this purpose in their interactions with learners, it should be included in faculty development initiatives, along with a discussion of the key importance of separating feedback from assessment, to promote a feedback culture of growth and support programmatic assessment of competence.

Levelling the playing field: students' motivations to contribute to an amnesty of assessment materials

BACKGROUND

'Exam recall' is a recognised phenomenon whereby students recall and record questions after leaving the examination hall. This poses two main problems. First, as these questions are only available to peers of the students who recall the questions, these individuals have an unfair advantage. Secondly, the distribution of these recalled questions poses a threat to the validity and defensibility of assessments. To address the first of these problems, we developed an amnesty enabling students to submit assessment material to an on-line site. This study sought to explore which factors influence students' contributions to an amnesty of assessment material.

METHODS

We conducted a qualitative study using semi-structured focus groups. We used convenience sampling and recruited participants from all years of our undergraduate medical programme. The focus groups were facilitated by a medical student peer to reduce the power imbalance and encourage participants to discuss candidly. The focus groups were audio recorded and transcribed verbatim. Two researchers independently analysed all transcripts using thematic analysis and the research team met regularly to discuss emergent findings. Nvivo was used to assist with thematic analysis of the transcripts.

RESULTS

Twenty-six individuals participated in six focus groups. Six themes were identified through the analysis, which were categorised into motivating factors and de-motivating factors. Motivating factors were a perception that this would overcome inequity, a fear of repercussions, and the perceived usefulness of resources. Factors that prevented students contributing were a culture of competition, a lack of incentives, and mistrust of the medical school.

CONCLUSIONS

The establishment of an amnesty was acceptable to students and they were motivated to contribute materials. The competitive nature of medical careers and the stakes of summative assessments meant that students felt that some peers might still not contribute their materials. Students felt that the school were listening to their concerns and this led to a better dialogue between students and faculty.

A paediatric logbook: Millstone or milestone?

Logbooks are ubiquitous in undergraduate and postgraduate medical education. Here, two alumni who are in the early phase of their career, reflect on their experience in their undergraduate paediatric term and how this was shaped by their mandatory paediatric logbook.

Are we generating more assessments without added value? Surgical trainees' perceptions of and receptiveness to cross-specialty assessment

INTRODUCTION

Competency-based medical education (CBME) hinges on robust assessment. However, integrating regular workplace-based assessment within demanding and sometimes chaotic clinical environments remains challenging. Many faculty lack assessment expertise, and some programs lack the infrastructure and faculty numbers to fulfill CBME's mandate. Recognizing this, we designed and implemented an assessment innovation that trains and deploys a cadre of faculty to assess in specialties outside their own. Specifically, we explored trainees' perceptions of and receptiveness to this novel assessment approach.

METHODS

Within Western University's Surgical Foundations program, 27 PGY‑1 trainees were formatively assessed by trained non-surgeons on a basic laparoscopic surgical skill. These assessments did not impact trainees' progression. Four focus groups were conducted to gauge residents' sentiments about the experience of cross-specialty assessment. Data were then analyzed using a thematic analysis approach.

RESULTS

While a few trainees found the experience motivating, more often trainees questioned the feedback they received and the practicality of this assessment approach to advance their procedural skill acquisition. What trainees wanted were strategies for improvement, not merely an assessment of performance.

DISCUSSION

Trainees' trepidation at the idea of using outside assessors to meet increased assessment demands appeared grounded in their expectations for assessment. What trainees appeared to desire was a coach-someone who could break their performance into its critical individual components-as opposed to an assessor whose role was limited to scoring their performance. Understanding trainees' receptivity to new assessment approaches is crucial; otherwise training programs run the risk of generating more assessments without added value.

Toward Practice-Based Continuing Education Protocols: Using Testing to Help Physicians Update Their Knowledge

INTRODUCTION

Using assessment to facilitate learning is a well-established priority in education but has been associated with variable effectiveness for continuing professional development. What factors modulate the impact of testing in practitioners are unclear. We aimed to improve capacity to support maintenance of competence by exploring variables that influence the value of web-based pretesting.

METHODS

Family physicians belonging to a practice-based learning program studied two educational modules independently or in small groups. Before learning sessions they completed a needs assessment and were assigned to either sit a pretest intervention or read a relevant review article. After the learning session, they completed an outcome test, indicated plans to change practice, and subsequently documented changes made.

RESULTS

One hundred twelve physicians completed the study, 92 in small groups. The average lag between tests was 6.3 weeks. Relative to those given a review article, physicians given a pretest intervention: (1) reported spending less time completing the assigned task (16.7 versus 25.7 minutes); (2) performed better on outcome test questions that were repeated from the pretest (65.9% versus 58.7%); and (3) when the learning module was completed independently, reported making a greater proportion of practice changes to which they committed (80.0% versus 45.0%). Knowledge gain was unrelated to physicians' stated needs.

DISCUSSION

Low-stakes formative quizzes, delivered with feedback, can influence the amount of material practicing physicians remember from an educational intervention independent of perceptions regarding the need to engage in continuing professional development on the particular topic.