Workplace-based assessment and students' approaches to learning: a qualitative inquiry

Implementation and Impact of an Adapted Digital Perioperative Competency-Building Tool to Enhance Teaching, Learning And Feedback in Postgraduate Competency-Based Medical Education

OBJECTIVES

The purpose of this educational intervention was to introduce, iteratively adapt, and implement a digital formative assessment tool in a surgical speciality. The study also evaluated the intervention's impact on perioperative teaching, learning, feedback, and surgical competency.

DESIGN

A participatory action research model with a mixed methods approach.

SETTING

This study was performed over 10 months in an institutional hospital in South Africa with a general surgery department.

PARTICIPANTS

Twelve supervising surgical trainers/faculty and 12 surgical trainees/residents consented to participate in the intervention.

RESULTS

The first 4 months of the intervention focused on relationship building, a multi-stakeholder contextual needs assessment and training sessions to support a shared mindset and shift in the teaching and learning culture. The final adapted perioperative competency-building tool comprised a 23-item assessment with four open-text answers (Table 1). Over the following 6-month period, 48 workplace-based competency-building perioperative evaluations were completed. Most trainees took less than 5 minutes to self-assess (67%) before most trainers (67%) took less than 5 minutes to give oral feedback to the trainee after the perioperative supervised learning encounter. On average, the digital tool took 6 minutes to complete during the bidirectional perioperative teaching and learning encounter with no negative impact on the operational flow. All trainers and trainees reported the training and implementation of the digital tool to be beneficial to teaching, learning, feedback, and the development of surgical competency. Analysis of the completed tools revealed several trainees showing evidence of progression in surgical competency for index procedures within the speciality. The focus groups and interviews also showed a change in the teaching and learning culture: more positively framed, frequent, structured, and specific feedback, improved accountability, and trainee-trainer perioperative readiness for teaching. Highlighted changes included the usefulness of trainee self-assessment before perioperative trainer feedback and the tool's value in improving competency to Kirkpatrick Level 4.

CONCLUSION

Implementing an adapted digital Workplace-Based Assessment (WBA) tool using a participatory action research model has proven successful in enhancing the effectiveness of supervised perioperative teaching and learning encounters. This approach has improved teaching and feedback practices, facilitated the development of surgical competency, and ultimately impacted the overall culture to Kirkpatrick level 4. Importantly, it has positively influenced the trainee-trainer relationship dynamic. Based on these positive outcomes, we recommend using this effective method and our relationship-centred framework for implementing formative competency-building tools in future studies. By doing so, larger-scale and successful implementation of Competency-Based Medical Education (CBME) could be achieved in various contexts. This approach can potentially enhance teaching and learning encounters, promote competency development, and improve the overall educational experience for surgical trainees and trainers.

Use of exemplars-based feedforward to improve the performance of medical students in constructed response assessment

Background

Exemplars are the graded responses to theory questions from previous examinations and are considered as means to guide students about what is valued by assessors; hence giving feed forward to students before the exam and developing self-regulated learning among students. The present study was undertaken to compare the effect on performance of students between intervention (didactic lecture and exemplar) and control group (didactic lecture).

Methods

This study was an interventional study conducted among 256 students of a medical college in Rajasthan, India. Participants were randomized to either the control group A or to the interventional group B. Outcomes included a post intervention test evaluating the knowledge gained after the session and students 'attitudes about the same. A focus group discussion was conducted investigating the perceptions. The feedback from faculty was also taken.

Results

The mean post test score of intervention group was found to be 6.1 ± 1.74 and of the control group was 5.5 ± 1.6; significantly higher with p-value of 0.03. Out of 128 participants in group B, majority (n = 94,73%) believed that exemplars improve learning and created their interest in topic (n = 78, 60.9%). Most of students (66%) were of the opinion that exemplars should be regularly used with didactic lectures. Key themes identified from focus group were: advantages of the exemplars-based feedforward, challenges in implementation and ways to improve the method.

Conclusion

Exemplars improved the performance of students. The faculty members and students agreed that they are an effective tool to enhance learning.

"We just don't have the resources": Supervisor perspectives on introducing workplace-based assessments into medical specialist training in South Africa

BACKGROUND

South Africa (SA) is on the brink of implementing workplace-based assessments (WBA) in all medical specialist training programmes in the country. Despite the fact that competency-based medical education (CBME) has been in place for about two decades, WBA offers new and interesting challenges. The literature indicates that WBA has resource, regulatory, educational and social complexities. Implementing WBA would therefore require a careful approach to this complex challenge. To date, insufficient exploration of WBA practices, experiences, perceptions, and aspirations in healthcare have been undertaken in South Africa or Africa. The aim of this study was to identify factors that could impact WBA implementation from the perspectives of medical specialist educators. The outcomes being reported are themes derived from reported potential barriers and enablers to WBA implementation in the SA context.

METHODS

This paper reports on the qualitative data generated from a mixed methods study that employed a parallel convergent design, utilising a self-administered online questionnaire to collect data from participants. Data was analysed thematically and inductively.

RESULTS

The themes that emerged were: Structural readiness for WBA; staff capacity to implement WBA; quality assurance; and the social dynamics of WBA.

CONCLUSIONS

Participants demonstrated impressive levels of insight into their respective working environments, producing an extensive list of barriers and enablers. Despite significant structural and social barriers, this cohort perceives the impending implementation of WBA to be a positive development in registrar training in South Africa. We make recommendations for future research, and to the medical specialist educational leaders in SA.

Designing and implementing an OMMID midwifery professional competence mixed test: A multimethod, multiphasic study

The assessment of professional competence is essential to ensure the achievement of competence standards. The goal of this study is to design and implement a professional competence test model to make the test unified, comprehensive, and fair. This research was a multimethod, multiphasic study. The first qualitative phase of the nominal group technique was conducted to design the test model with specific guidelines. The second phase was a mixed-method parallel field trial conducted on 161 senior midwifery bachelor students in universities of Iran. The test was conducted following the traditional method in the control group and OMMID model in the intervention group. This model consists of three parts: the MCQs (multiple choice questions), OSCE (objective structured clinical examination), and clinically-oriented tests, which included Mini-CEX (mini-clinical evaluation exercise) and DOPS (direct observation of procedural skills). Data were collected using questionnaires and interviews, and the results were assessed qualitatively and quantitatively. The OMMID model and the associated guideline were designed. Qualitative data analysis resulted in six main themes, including organizing, structure, tension due to change, fairness, unification, and outcome. The OMMID model did not increase students' anxiety and stress and did not decrease their satisfaction. The merits of this model include having a centralized guideline, using multiple evaluation methods, comprehensive evaluation of necessary skills, promotion of fairness, and increased student satisfaction.

Beyond Checking Boxes: Exploring Tensions With Use of a Workplace-Based Assessment Tool for Formative Assessment in Clerkships

PURPOSE

To understand the role of a workplace-based assessment (WBA) tool in facilitating feedback for medical students, this study explored changes and tensions in a clerkship feedback activity system through the lens of cultural historical activity theory (CHAT) over 2 years of tool implementation.

METHOD

This qualitative study uses CHAT to explore WBA use in core clerkships by identifying feedback activity system elements (e.g., community, tools, rules, objects) and tensions among these elements. University of California, San Francisco core clerkship students were invited to participate in semistructured interviews eliciting experience with a WBA tool intended to enhance direct observation and feedback in year 1 (2019) and year 2 (2020) of implementation. In year 1, the WBA tool required supervisor completion in the school's evaluation system on a computer. In year 2, both students and supervisors had WBA completion abilities and could access the form via a smartphone separate from the school's evaluation system.

RESULTS

Thirty-five students participated in interviews. The authors identified tensions that shifted with time and tool iterations. Year 1 students described tensions related to cumbersome tool design, fear of burdening supervisors, confusion over WBA purpose, WBA as checking boxes, and WBA usefulness depending on clerkship context and culture. Students perceived dissatisfaction with the year 1 tool version among peers and supervisors. The year 2 mobile-based tool and student completion capabilities helped to reduce many of the tensions noted in year 1. Students expressed wider WBA acceptance among peers and supervisors in year 2 and reported understanding WBA to be for low-stakes feedback, thereby supporting formative assessment for learning.

CONCLUSIONS

Using CHAT to explore changes in a feedback activity system with WBA tool iterations revealed elements important to WBA implementation, including designing technology for tool efficiency and affording students autonomy to document feedback with WBAs.

Factors that may improve paediatric workplace-based assessments: an exploratory study

OBJECTIVES

To establish if paediatric trainees are satisfied with the current workplace-based assessment (WBA) process. To identify factors that contribute both positively and negatively to the educational experience during WBAs. To find out if trainees and their supervisors experience any challenges conducting WBAs. To establish potential ways to improve future assessments.

DESIGN

Qualitative semistructured interviews.

SETTING

Participants included fifteen trainees (ST1-8) in general paediatric and subspecialty posts and four consultants or equivalent across five hospital sites in the Thames Valley Deanery. All participants had regular exposure to WBAs.

INTERVENTIONS

Interviews were undertaken between June 2020 and January 2021 via video link. Data collection and analysis were conducted iteratively using constant comparison until theoretical sufficiency was achieved.

MAIN OUTCOME MEASURE

Using Constructivist Grounded Theory, a theoretical framework, grounded in the data, was developed that depicted the core elements that should be present to optimise WBAs.

RESULTS

A number of key components were reported to affect the educational value of WBAs. A positive departmental culture towards education and training is essential. Chosen cases should be challenging, and direct observation or in-depth discussion, depending on the assessment type, is fundamental. Timely constructive feedback and immediate completion of the assessment form are also imperative.

CONCLUSION

Some trainees experienced WBAs where these key components aligned, but many did not, and this negatively affected their learning. Three main challenges or future targets for further improvements include increasing time, improving training and optimising technology.

Hyperglobalist, sceptical, and transformationalist perspectives on globalization in medical education

PURPOSE

Globalisation has been hotly debated in recent decades and has seemingly had a profound impact on medical education. This review synthesises the medical education literature using key perspectives from globalisation theory by Holton (Making globalisation).

METHODS

Holton (Making globalisation) recognised three key perspectives in globalisation theory-hyperglobalist, sceptical, and transformationalist. This article critically reviews the literature on globalisation in the field of medical education using this theoretical framework.

RESULTS

Hyperglobalist and sceptical perspectives dominated early periods of medical education literature on globalisation, projecting it either as a mainly positive or mainly negative force, respectively. Most forecasts grounded in these perspectives have not materialised in medical education policy and practice. Since 2010, the volume of scholarship about globalisation has increased and has been predominantly transformationalist in perspective, recognising a reality that has both positive and negative consequences.

CONCLUSIONS

The medical education literature has mirrored the broader social science literature, in moving over time from hyperglobalist and sceptical positions, towards a 'third wave' of globalisation thinking that is transformationalist. Medical education practitioners and policymakers should be mindful of these perspectives and trends as they navigate the opportunities and challenges presented by globalisation.

Moving toward Mastery: Changes in Student Perceptions of Clerkship Assessment with Pass/Fail Grading and Enhanced Feedback

ProblemClerkship grades contribute to a summative assessment culture in clerkships and can therefore interfere with students' learning. For example, by focusing on summative, tiered clerkship grades, students often discount accompanying feedback that could inform future learning. This case report seeks to explore whether an assessment system intervention which eliminated tiered grades and enhanced feedback was associated with changes in student perceptions of clerkship assessment and perceptions of the clinical learning environment. Intervention: In January 2019, our institution eliminated tiered clerkship grading (honors/pass/fail) for medical students during the core clerkship year and implemented pass/fail clerkship grading along with required twice weekly, work-based assessments for formative feedback. Context: In this single institution, cross-sectional survey study, we collected data from fourth-year medical students one year after an assessment system intervention. The intervention entailed changing from honors/pass/fail to pass/fail grading in all eight core clerkships and implementing an electronic system to record twice-weekly real-time formative work-based assessments. The survey queried student perceptions on the fairness and accuracy of grading and the clinical learning environment-including whether clerkships were mastery- or performance-oriented. We compared responses from students one year after the assessment intervention to those from the class one year before the intervention. Comparisons were made using unpaired, two-tailed t-tests or chi-squared tests as appropriate with Cohen's d for effect size estimation for score differences. Content analysis was used to analyze responses from two open-ended questions about feedback and grading. Impact: Survey response rates were similar before and after intervention (76% (127/168) vs. 72% (118/163), respectively) with no between-group differences in demographics. The after-intervention group showed statistically significant increases in the following factors: "grades are transparent and fair" (Cohen's d = 0.80), "students receive useful feedback" (d = 0.51), and "resident evaluation procedures are fair" (d = 0.40). After-intervention respondents perceived the clerkship learning environment to be more mastery-oriented (d = 0.52), less performance approach-oriented (d = 0.63), and less performance avoid-oriented (d = 0.49). There were no statistical differences in the factors "attending evaluation procedures are fair," "evaluations are accurate," "evaluations are biased," or "perception of stereotype threat." Open-ended questions revealed student recommendations to improve clerkship summary narratives, burden of work-based assessment, and in-person feedback. Lessons Learned: After an assessment system change to pass/fail grading with work-based assessments, we observed moderate to large improvements in student perceptions of clerkship grading and the mastery orientation of the learning environment. Our intervention did not improve perceptions around bias in assessment in clerkships. Other medical schools may consider similar interventions to begin to address student concerns with clerkship assessment and promote a more adaptive learning environment.

Implementation and Use of Workplace-Based Assessment in Clinical Learning Environments: A Scoping Review

PURPOSE

Workplace-based assessment (WBA) serves a critical role in supporting competency-based medical education (CBME) by providing assessment data to inform competency decisions and support learning. Many WBA systems have been developed, but little is known about how to effectively implement WBA. Filling this gap is important for creating suitable and beneficial assessment processes that support large-scale use of CBME. As a step toward filling this gap, the authors describe what is known about WBA implementation and use to identify knowledge gaps and future directions.

METHOD

The authors used Arksey and O'Malley's 6-stage scoping review framework to conduct the review, including: (1) identifying the research question; (2) identifying relevant studies; (3) study selection; (4) charting the data; (5) collating, summarizing, and reporting the results; and (6) consulting with relevant stakeholders.

RESULTS

In 2019-2020, the authors searched and screened 726 papers for eligibility using defined inclusion and exclusion criteria. One hundred sixty-three met inclusion criteria. The authors identified 5 themes in their analysis: (1) Many WBA tools and programs have been implemented, and barriers are common across fields and specialties; (2) Theoretical perspectives emphasize the need for data-driven implementation strategies; (3) User perceptions of WBA vary and are often dependent on implementation factors; (4) Technology solutions could provide useful tools to support WBA; and (5) Many areas of future research and innovation remain.

CONCLUSIONS

Knowledge of WBA as an implemented practice to support CBME remains constrained. To remove these constraints, future research should aim to generate generalizable knowledge on WBA implementation and use, address implementation factors, and investigate remaining knowledge gaps.

Framework for curriculum delivery during COVID-19 pandemic in a health sciences university

This article aims to shed light on the management that was taken by the King Saud Bin Abdulaziz University for Health Sciences to accommodate the immediate needs for online curriculum delivery, in response to the total lockdown due to COVID-19 pandemic. We have described the process done, actions implemented, and challenges faced to manage the curriculum delivery during the pandemic and to plan the subsequent year curriculum delivery. Effective management will be enhanced by focused faculty development, curriculum management, assessment planning, and technical support. We believe that the management done can be taken as a model in similar situations where sudden online curriculum delivery is deemed necessary. Further audit on the effectiveness and implication of these actions is required after the end of the pandemic.

Exploring the Construct of Psychological Safety in Medical Education

PURPOSE

Psychological safety (PS) is recognized as key in health professions education. However, most studies exploring PS in medical education have focused on mistreatment, thus focusing on what PS is not. The authors set out to explicitly explore learners' concept of PS in the context of medical education to better understand and define PS and its educational consequences for medical students.

METHOD

This descriptive exploratory study was conducted in the context of a pilot peer-assisted learning (PAL) program. The program brought together residents and medical students for 16 semiformal learning sessions. Eight medical students from a PAL program were recruited for semistructured interviews to explore their experiences of PS. Transcripts were thematically analyzed using an inductive approach, and social ecological theory was integrated in the later stages of analysis.

RESULTS

Students described PS as not feeling judged. Having supportive relationships with peers and mentors improved PS. Students' sense of PS appeared to free them to focus on learning in the present moment without considering the consequences for their image in the eyes of others. Feeling safe also seemed to facilitate relationship building with the mentors.

CONCLUSIONS

A sense of PS appears to free learners from constantly being self-conscious about projecting an image of competence. This enables learners to be present in the moment and concentrate on engaging with the learning task at hand. The authors propose that the term "educational safety" be used to describe a relational construct that can capture the essence of what constitutes PS for learners.

Educational impact of assessment on medical students' learning at Tehran University of Medical Sciences: a qualitative study

OBJECTIVES

It has been shown that assessment strongly affects students' performance. A deeper insight needs to be gained into the interplay of assessment and learning. The aim of the current study was to develop a model to explain the educational impact of assessments on students' learning, before, during and after the test.

DESIGN

This study used semistructured interviews, focus group discussions and observation and collection of field notes. A qualitative methodology using the grounded theory data analysis approach was then used to generate an explanation of the process of how assessment impacts students' learning.

SETTING

School of Medicine, Tehran University of Medical Sciences.

PARTICIPANTS

Participants were medical students and teachers with first-hand experience or expertise in assessment as well as their willingness to participate in the study. Fifteen people (eight medical students, seven faculty members) were interviewed. One focus group discussion (with five students) was held.

RESULTS

The extracted concepts from our study were classified into four main categories. These categories include elements of the assessment programme which affect learning, the mechanism through which they exert their effects, contextual factors and the impact they have on learning. These elements and their interplay occur within an environment with its antecedent characteristics.

CONCLUSIONS

This study suggested a model for understanding the elements of the assessment which, within the context, affect learning, the mechanisms through which they impart their effects and the final outcomes obtained.

Student feedback experiences in a cross-border medical education curriculum

OBJECTIVES

To determine non-Western situated health professional student experiences and preferences for feedback in workplace-based settings.

METHODS

We conducted five focus groups with 27 students of Arab-origin enrolled in a Canadian-accredited cross-border pharmacy program in Qatar. Transcripts of recorded discussions were analyzed using the framework method.  Hofstede's and Hall's cultural dimension models were employed to understand described feedback encounters and behaviours.

RESULTS

We identified three themes associated with cultural influences on student feedback experiences, namely: 1) collectivism; 2) power distance; and 3) context.  Trainees described clinical supervisors who inadequately recognized individual performance, rejected critique, and insufficiently documented feedback onto the written in-training evaluation report. Conversely, students expected specific and timely feedback, invited criticism for learning, and desired clear written commentary.

CONCLUSIONS

Feedback behaviours of clinical supervisors, but not those of trainees, were consistent with local cultural norms as described by Hofstede and Hall.  Instead, feedback expectations of pharmacy students in Qatar largely echo those of other trainees enrolled in professional curricula situated outside the Middle East. Principles for optimal feedback in clinical training largely arise from Western perspectives but are not necessarily universal. Our work demonstrates that practices, in part, may be subject to local socio-cultural influences.  This is of particular importance in the experiential training component of cross-border medical education programs adopted by overseas institutions. Our findings also further add to the growing body of literature reporting suboptimal feedback in workplace-based learning, reinforcing the need to cultivate more student-centered practices in health professional training globally.

Navigating the system: Physiotherapy student perceptions of performance-based assessment

BACKGROUND

Performance-based assessment (PBA) is an integral component of health professional education as it determines students' readiness for independent practice. Stakeholder input can provide valuable insight regarding its challenges, facilitators, and impact on student learning, which may further its evolution. Currently, evidence of stakeholder opinion is limited. Thus, we aimed to explore physiotherapy students' perceptions of performance-based assessment in their capacity as its central stakeholders.

METHODS

A qualitative interpretive constructivist approach was employed using focus group interviews for data collection. Six focus groups were completed (n = 33). Inductive thematic analysis was used to explore the data.

RESULTS

Two themes were identified. The first outlined perceived inconsistencies within the process, and how these impacted on student learning. The second described how students used their experiential knowledge to identify strategies to manage these challenges thus identifying key areas for improvement.

CONCLUSION

Inconsistencies outlined within the current physiotherapy performance-based assessment process encourage an emphasis on grades rather than on learning. It is timely that the physiotherapy academic and clinical communities consider these findings alongside evidence from other health professions to improve assessment procedures and assure public confidence and patient safety.

Assessment in a global context: An international perspective on dental education

Assessments are widely used in dental education to record the academic progress of students and ultimately determine whether they are ready to begin independent dental practice. Whilst some would consider this a "rite-of-passage" of learning, the concept of assessments in education is being challenged to allow the evolution of "assessment for learning." This serves as an economical use of learning resources whilst allowing our learners to prove their knowledge and skills and demonstrating competence. The Association for Dental Education in Europe and the American Dental Education Association held a joint international meeting in London in May 2017 allowing experts in dental education to come together for the purposes of Shaping the Future of Dental Education. Assessment in a Global Context was one topic in which international leaders could discuss different methods of assessment, identifying the positives, the pitfalls and critiquing the method of implementation to determine the optimum assessment for a learner studying to be a healthcare professional. A post-workshop survey identified that educators were thinking differently about assessment, instead of working as individuals providing isolated assessments; the general consensus was that a longitudinally orientated systematic and programmatic approach to assessment provide greater reliability and improved the ability to demonstrate learning.

Guidelines: The do's, don'ts and don't knows of direct observation of clinical skills in medical education

INTRODUCTION

Direct observation of clinical skills is a key assessment strategy in competency-based medical education. The guidelines presented in this paper synthesize the literature on direct observation of clinical skills. The goal is to provide a practical list of Do's, Don'ts and Don't Knows about direct observation for supervisors who teach learners in the clinical setting and for educational leaders who are responsible for clinical training programs.

METHODS

We built consensus through an iterative approach in which each author, based on their medical education and research knowledge and expertise, independently developed a list of Do's, Don'ts, and Don't Knows about direct observation of clinical skills. Lists were compiled, discussed and revised. We then sought and compiled evidence to support each guideline and determine the strength of each guideline.

RESULTS

A final set of 33 Do's, Don'ts and Don't Knows is presented along with a summary of evidence for each guideline. Guidelines focus on two groups: individual supervisors and the educational leaders responsible for clinical training programs. Guidelines address recommendations for how to focus direct observation, select an assessment tool, promote high quality assessments, conduct rater training, and create a learning culture conducive to direct observation.

CONCLUSIONS

High frequency, high quality direct observation of clinical skills can be challenging. These guidelines offer important evidence-based Do's and Don'ts that can help improve the frequency and quality of direct observation. Improving direct observation requires focus not just on individual supervisors and their learners, but also on the organizations and cultures in which they work and train. Additional research to address the Don't Knows can help educators realize the full potential of direct observation in competency-based education.

Medical student changes in self-regulated learning during the transition to the clinical environment

BACKGROUND

Self-regulated learning (SRL), which is learners' ability to proactively select and use different strategies to reach learning goals, is associated with academic and clinical success and life-long learning. SRL does not develop automatically in the clinical environment and its development during the preclinical to clinical learning transition has not been quantitatively studied. Our study aims to fill this gap by measuring SRL in medical students during the transitional period and examining its contributing factors.

METHODS

Medical students were invited to complete a questionnaire at the commencement of their first clinical year (T0), and 10 weeks later (T1). The questionnaire included the Motivated Strategies for Learning Questionnaire (MSLQ) and asked about previous clinical experience. Information about the student's background, demographic characteristics and first clinical rotation were also gathered.

RESULTS

Of 118 students invited to participate, complete paired responses were obtained from 72 medical students (response rate 61%). At T1, extrinsic goal orientation increased and was associated with gender (males were more likely to increase extrinsic goal orientation) and type of first attachment (critical care and community based attachments, compared to hospital ward based attachments). Metacognitive self-regulation decreased at T1 and was negatively associated with previous clinical experience.

CONCLUSIONS

Measurable changes in self-regulated learning occur during the transition from preclinical learning to clinical immersion, particularly in the domains of extrinsic goal orientation and metacognitive self-regulation. Self-determination theory offers possible explanations for this finding which have practical implications and point the way to future research. In addition, interventions to promote metacognition before the clinical immersion may assist in preserving SRL during the transition and thus promote life-long learning skills in preparation for real-world practice.

How does feedback in mini-CEX affect students' learning response?

OBJECTIVE

This study was aimed to explore students' learning response toward feedback during mini-CEX encounter.

METHODS

This study used a phenomenological approach to identify the students' experiences toward feedback during mini-CEX encounter. Data was collected using Focus Group Discussion (FGD) for all students who were in their final week of clerkship in the internal medicine rotation. There were 4 FGD groups (6 students for each group). All FGD were audio-taped and transcribed verbatim. The FGD transcripts were analyzed thematically and managed using Atlas-ti (version 7.0).

RESULTS

Feedback content and the way of providing feedback on mini-CEX stimulated students' internal process, including self-reflection, emotional response, and motivation. These internal processes encouraged the students to take action or do a follow-up on the feedback to improve their learning process. In addition, there was also an external factor, namely consequences, which also influenced the students' reaction to the follow-up on feedback. In the end, this action caused several learning effects that resulted in the students' increased self-efficacy, attitude, knowledge and clinical skill.

CONCLUSIONS

Feedback content and the way of providing feedback on mini-CEX stimulates the students' internal processes to do a follow-up on feedback. However, another external factor also affects the students' decision on the follow-up actions. The follow-ups result in various learning effects on the students. Feedback given along with summative assessment enhances learning effects on students, as well. It is suggested that supervisors of clinical education are prepared to comprehend every factor influencing feedback on mini CEX to improve the students' learning response.

'I wish someone watched me interview:' medical student insight into observation and feedback as a method for teaching communication skills during the clinical years

BACKGROUND

Experts suggest observation and feedback is a useful tool for teaching and evaluating medical student communication skills during the clinical years. Failing to do this effectively risks contributing to deterioration of students' communication skills during the very educational period in which they are most important. While educators have been queried about their thoughts on this issue, little is known about what this process is like for learners and if they feel they get educational value from being observed. This study explored student perspectives regarding their experiences with clinical observation and feedback on communication skills.

METHODS

A total of 125 senior medical students at a U.S. medical school were interviewed about their experiences with observation and feedback. Thematic analysis of interview data identified common themes among student responses.

RESULTS

The majority of students reported rarely being observed interviewing, and they reported receiving feedback even less frequently. Students valued having communication skills observed and became more comfortable with observation the more it occurred. Student-identified challenges included supervisor time constraints and grading based on observation. Most feedback focused on information gathering and was commonly delayed until well after the observed encounter.

CONCLUSIONS

Eliciting students' perspectives on the effect of observation and feedback on the development of their communication skills is a unique way to look at this topic, and brings to light many student-identified obstacles and opportunities to maximize the educational value of observation and feedback for teaching communication, including increasing the number of observations, disassociating observation from numerically scored evaluation, training faculty to give meaningful feedback, and timing the observation/feedback earlier in clerkships.

Beyond homogenization discourse: Reconsidering the cultural consequences of globalized medical education

INTRODUCTION

Global medical education standards, largely designed in the West, have been promoted across national boundaries with limited regard for cultural differences. This review aims to identify discourses on cultural globalization in medical education literature from non-Western countries.

METHODS

To explore the diversity of discourses related to globalization and culture in the field of medical education, the authors conducted a critical review of medical education research from non-Western countries published in Academic Medicine, Medical Education and Medical Teacher from 2006 to 2014. Key discourses about globalization and culture emerged from a preliminary analysis of this body of literature. A secondary analysis identified inductive sub-themes.

RESULTS

Homogenization, polarization and hybridization emerged as key themes in the literature. These findings demonstrate the existence of discourses beyond Western-led homogenization and the co-existence of globalization discourses ranging from homogenization to syncretism to resistance.

CONCLUSIONS

This review calls attention to the existence of manifold discourses about globalization and culture in non-Western medical education contexts. In refocusing global medical education processes to avoid Western cultural imperialism, it will also be necessary to avoid the pitfalls of other globalization discourses. Moving beyond existing discourses, researchers and educators should work towards equitable, context-sensitive and locally-driven approaches to global medical education.

Workplace-based assessment: a review of user perceptions and strategies to address the identified shortcomings

Workplace based assessments (WBAs) are now commonplace in postgraduate medical training. User acceptability and engagement is essential to the success of any medical education innovation. To this end, possessing an insight into trainee and trainer perceptions towards WBAs will help identify the major problems, permitting strategies to be introduced to improve WBA implementation. A review of literature was performed to identify studies examining trainee and trainer perceptions towards WBAs. Studies were excluded if non-English or sampling a non-medical/dental population. The identified literature was synthesised for the purpose of this critical narrative review. It is clear that there is widespread negativity towards WBAs in the workplace. This has negatively impacted on the effectiveness of WBA tools as learning aids. This negativity exists in trainees but also to an extent in their trainers. Insight gained from the literature reveals three dominant problems with WBA implementation: poor understanding as to the purpose of WBAs; insufficient time available for undertaking these assessments; and inadequate training of trainers. Approaches to addressing these three problems with WBA implementation are discussed. It is likely that a variety of solutions will be required. The prevalence of negativity towards WBAs is substantial in both trainees and trainers, eroding the effectiveness of learning that is consequent upon them. The educational community must now listen to the concerns being raised by the users and consider the range of strategies being proposed to improve the experiences of trainees, and their trainers.

Same-level peer-assisted learning in medical clinical placements: a narrative systematic review

BACKGROUND

Peer-assisted learning (PAL) is increasingly used in medical education, and the benefits of this approach have been reported. Previous reviews have focused on the benefits of peer tutoring of junior students by senior students. Forms of PAL such as discussion groups and role-playing have been neglected, as have alternative teacher-learner configurations (e.g. same-level PAL) and the effects on other stakeholders, including clinician educators and patients. This review examines the benefits of same-level PAL for students, clinician educators and patients in pre-registration clinical medical education.

METHODS

Medline, PsycINFO, CINAHL and ERIC were searched in March 2014. A total of 1228 abstracts were retrieved for review; 64 full-text papers were assessed. Data were extracted from empirical studies describing a same-level PAL initiative in a clinical setting, focusing on effects beyond academic performance and student satisfaction. Qualitative thematic analysis was employed to identify types of PAL and to cluster the reported PAL effects.

RESULTS

Forty-three studies were included in the review. PAL activities were categorised into role-play, discussion, teaching and assessment. Only 50% of studies reported information beyond self-report and satisfaction with the PAL intervention. Benefits for students (including development of communication and professional skills) and clinician educators (developing less-used facilitation skills) were reported. Direct patient outcomes were not identified. Caveats to the use of PAL emerged, and guidelines for the use of PAL were perceived as useful.

CONCLUSION

Many student-related benefits of PAL were identified. PAL contributes to the development of crucial skills required for a doctor in the workplace. Vertical integration of learning and teaching skills across the curriculum and tools such as feedback checklists may be required for successful PAL in the clinical environment. Benefits for patients and educators were poorly characterised within the included studies. Future work should evaluate the use of PAL with regards to student, clinician educator and patient outcomes.

The impact of programmatic assessment on student learning: theory versus practice

CONTEXT

It is widely acknowledged that assessment can affect student learning. In recent years, attention has been called to 'programmatic assessment', which is intended to optimise both learning functions and decision functions at the programme level of assessment, rather than according to individual methods of assessment. Although the concept is attractive, little research into its intended effects on students and their learning has been conducted.

OBJECTIVES

This study investigated the elements of programmatic assessment that students perceived as supporting or inhibiting learning, and the factors that influenced the active construction of their learning.

METHODS

The study was conducted in a graduate-entry medical school that implemented programmatic assessment. Thus, all assessment information, feedback and reflective activities were combined into a comprehensive, holistic programme of assessment. We used a qualitative approach and interviewed students (n = 17) in the pre-clinical phase of the programme about their perceptions of programmatic assessment and learning approaches. Data were scrutinised using theory-based thematic analysis.

RESULTS

Elements from the comprehensive programme of assessment, such as feedback, portfolios, assessments and assignments, were found to have both supporting and inhibiting effects on learning. These supporting and inhibiting elements influenced students' construction of learning. Findings showed that: (i) students perceived formative assessment as summative; (ii) programmatic assessment was an important trigger for learning, and (iii) the portfolio's reflective activities were appreciated for their generation of knowledge, the lessons drawn from feedback, and the opportunities for follow-up. Some students, however, were less appreciative of reflective activities. For these students, the elements perceived as inhibiting seemed to dominate the learning response.

CONCLUSIONS

The active participation of learners in their own learning is possible when learning is supported by programmatic assessment. Certain features of the comprehensive programme of assessment were found to influence student learning, and this influence can either support or inhibit students' learning responses.

Lessons learnt from a pilot of assessment for learning

A 12-month pilot was carried out on assessments for learning and assessments of learning as part of workplace-based assessments in postgraduate medical education. This was carried out in three regions and core medical trainees and higher specialty medical trainees participated. Focus groups and questionnaires were utilised to investigate the trainees' and trainers' experiences and perceptions of assessments for learning. The study demonstrated that the trainees and trainers perceived the newly introduced assessments for learning--supervised learning events (SLEs)--as learning tools. However, SLEs were often undertaken with no previous organisation and with no direct observation, regardless of the underlying purposes and methods of the WPBAs. There was a lack of, or delayed or non-specific, feedback following SLEs, which would have impeded its educational value. Trainee and trainer disengagement was one of the contributing factors. These findings are valuable in informing and facilitating future successful implementation of assessments for learning.

Workplace-based assessment for vocational registration of international medical graduates

INTRODUCTION

Medical regulatory authorities need efficient and effective methods of ensuring the competence of immigrating international medical graduates (IMGs). Not all IMGs who apply for specialist vocational registration will have directly comparable qualifications to those usually accepted. As general licensure examinations are inappropriate for these doctors, workplace-based assessment (WBA) techniques would appear to provide a solution. However, there is little published data on such outcomes.

METHODS

All cases of WBA (n = 81) used for vocational registration of IMGs in New Zealand between 2008 and 2013 were collated and analyzed.

RESULTS

The successful completion rate of IMGs through the pathway was 87%. The majority (64%) undertook the year of supervised practice and the final assessment in a provincial center. For those unsuccessful in the pathway, inadequate clinical knowledge was the most common deficit found, followed by poor clinical reasoning.

DISCUSSION

A WBA approach for assessing readiness of IMGs for vocational registration is feasible. The constructivist theoretical perspective of WBA has particular advantages in assessing the standard of practice for experienced practitioners working in narrow scopes than traditional methods of assessment. The majority of IMGs undertook both the clinical year and the assessment in provincial hospitals, thus providing a workforce for underserved areas.