Do portfolios have a future?

Candid insights and overlooked facets: what medical students write about patient-centeredness in diaries on longitudinal patient contacts

Despite students' exposure to patient-centered care principles, their dedication to patient-centeredness often experiences a wane throughout their academic journey. The process of learning patient-centeredness is complex and not yet fully understood. Therefore, in our study, we sought to explore what aspects of patient-centeredness students spontaneously document in their diaries during interactions with actual patients. This investigation will help to identify gaps in the current educational practices and better prepare future clinicians to deliver patient-centered healthcare. We analyzed 92 diaries of 28 third-year undergraduate medical students at UMC Utrecht in the Netherlands who participated in an educational intervention, following four patients each as companions over a two-year period early in their clerkships. We conducted thematic analysis, using inductive and deductive coding, within a social-constructionist paradigm. We identified four key themes: communication, the person behind the patient, collaboration and organization in healthcare, and students' professional development. Within these themes, we observed that students spontaneously documented 9 of 15 dimensions of patient-centeredness as outlined in the model of Scholl : 'clinician-patient communication', 'patient as unique person', 'biopsychological perspective', 'essential characteristics of the clinician', 'clinician-patient relationship', 'involvement of family and friends', 'patient-information', 'emotional support' and 'coordination and continuity of care' (mainly principles of patient-centeredness). Conversely, we noted that students underreported six other dimensions (enablers and activities): 'access to care', 'integration of medical and non-medical care', 'teamwork and teambuilding', 'patient involvement in care', 'patient empowerment' and 'physical support'. Throughout their longitudinal journey of following patients as non-medical companions, students spontaneously documented some aspects of patient-centeredness in their diaries. Additionally, students reflected on their own professional development. Our findings suggest that incorporating education on the broadness of the concept of patient-centeredness coupled with enhanced guidance, could potentially enable students to learn about the complete spectrum of patient-centeredness within their medical education.

Discoveries or doubts: a qualitative study of the transformative potential of portfolio meetings

To adapt to medical school, students need to change their approaches to learning and study. Transformative learning through critical reflection on disorienting learning experiences supports perspective change to direct new activity. We explored how portfolio meetings support changes in students' perspectives towards learning and study during the transition to studying medicine. This qualitative mixed methods study explored changes in medical students' perspectives before and after two portfolio meetings with a mentor during the first year of medicine. Adopting a constructionist approach, we analyzed interview transcripts and written reflections from a diverse sample of students using reflexive thematic analysis. The findings were integrated during analysis. Transformative learning theory was our interpretive lens. Our analysis revealed five themes. Two themes represented students' initial apprehension about portfolio meetings: The Disclosure Dilemma and A Question of Priorities. The theme "Seeing the Big Picture" described new perspectives from reflection on learning while preparing for meetings. Clarity from Dialogue described changed perspectives to learning and study after meetings. In the theme Dialogue Disappointment, meetings perceived as unhelpful led to persisting doubts about the value of portfolio meetings. Transformative learning was evident when students described new insights into their learning leading to goal-setting and new study strategies. When initial meetings were helpful, doubts about portfolio meetings were dispelled, enhancing student engagement in future meetings. Not all meetings were transformative, highlighting the importance of student and mentor preparation and training. Further research is needed to determine whether early portfolio experiences shape later engagement in clinical contexts.

Twelve tips for designing and implementing an academic coaching program

Coaching has become increasingly popular as a mechanism to support learning across the health professions education (HPE) continuum. While there is a growing body of literature in this area, there is minimal guidance related to the design and implementation of academic coaching in health professional courses. This paper seeks to contribute to this literature by presenting guidance for academic developers who are considering introducing academic coaching into a health professional course. The 12 tips are based on the authors' collective experiences of designing and implementing academic coaching in university medical courses in Australia and the UK. Although focused on medical education, this paper is intended to have applicability across the health professions, and potentially across university and postgraduate training contexts. Together, the tips offer a strategic and operational framework to guide the design and implementation of academic coaching initiatives in health professions education.

Evidence of learning in workplace-based assessments in a Family Medicine Training Programme

BACKGROUND

 Learning portfolios (LPs) provide evidence of workplace-based assessments (WPBAs) in clinical settings. The educational impact of LPs has been explored in high-income countries, but the use of portfolios and the types of assessments used for and of learning have not been adequately researched in sub-Saharan Africa. This study investigated the evidence of learning in registrars' LPs and the influence of the training district and year of training on assessments.

METHODS

 A cross-sectional study evaluated 18 Family Medicine registrars' portfolios from study years 1-3 across five decentralised training sites affiliated with the University of the Witwatersrand. Descriptive statistics were calculated for the portfolio and quarterly assessment (QA) scores and self-reported clinical skills competence levels. The competence levels obtained from the portfolios and university records served as proxy measures for registrars' knowledge and skills.

RESULTS

 The total LP median scores ranged from 59.9 to 81.0, and QAs median scores from 61.4 to 67.3 across training years. The total LP median scores ranged from 62.1 to 83.5 and 62.0 to 67.5, respectively in QAs across training districts. Registrars' competence levels across skill sets did not meet the required standards. Higher skills competence levels were reported in the women's health, child health, emergency care, clinical administration and teaching and learning domains.

CONCLUSION

 The training district and training year influence workplace-based assessment (WPBA) effectiveness. Ongoing faculty development and registrar support are essential for WPBA.Contribution: This study contributes to the ongoing discussion of how to utilise WPBA in resource-constrained sub-Saharan settings.

The identification of requirements for competency development during work-integrated learning in healthcare education

BACKGROUND

Work-integrated learning (WIL) is widely accepted and necessary to attain the essential competencies healthcare students need at their future workplaces. Yet, competency-based education (CBE) remains complex. There often is a focus on daily practice during WIL. Hereby, continuous competency development is at stake. Moreover, the fact that competencies need to continuously develop is often neglected.

OBJECTIVES

To ultimately contribute to the optimization of CBE in healthcare education, this study aimed at examining how competency development during WIL in healthcare education could be optimized, before and after graduation.

METHODS

Fourteen semi-structured interviews with 16 experts in competency development and WIL were carried out. Eight healthcare disciplines were included namely associate degree nursing, audiology, family medicine, nursing (bachelor), occupational therapy, podiatry, pediatrics, and speech therapy. Moreover, two independent experts outside the healthcare domain were included to broaden the perspectives on competency development. A qualitative research approach was used based on an inductive thematic analysis using Nvivo12© where 'in vivo' codes were clustered as sub-themes and themes.

RESULTS

The analysis revealed eight types of requirements for effective and continuous competency development, namely requirements in the context of (1) competency frameworks, (2) reflection and feedback, (3) assessment, (4) the continuity of competency development, (5) mentor involvement, (6) ePortfolios, (7) competency development visualizations, and (8) competency development after graduation. It was noteworthy that certain requirements were fulfilled in one educational program whereas they were absent in another. This emphasizes the large differences in how competence-based education is taking shape in different educational programs and internship contexts. Nevertheless, all educational programs seemed to recognize the importance of ongoing competency development.

CONCLUSION

The results of this study indicate that identifying and meeting the requirements for effective and continuous competency development is essential to optimize competency development during practice in healthcare education.

A systematic scoping review of group reflection in medical education

BACKGROUND

Reviewing experiences and recognizing the impact of personal and professional views and emotions upon conduct shapes a physician's professional and personal development, molding their professional identity formation (PIF). Poor appreciation on the role of reflection, shortages in trained tutors and inadequate 'protected time' for reflections in packed medical curricula has hindered its integration into medical education. Group reflection could be a viable alternative to individual reflections; however, this nascent practice requires further study.

METHODS

A Systematic Evidence Based Approach guided Systematic Scoping Review (SSR in SEBA) was adopted to guide and structure a review of group reflections in medical education. Independent searches of articles published between 1st January 2000 and 30th June 2022 in bibliographic and grey literature databases were carried out. Included articles were analysed separately using thematic and content analysis, and combined into categories and themes. The themes/categories created were compared with the tabulated summaries of included articles to create domains that framed the synthesis of the discussion.

RESULTS

1141 abstracts were reviewed, 193 full-text articles were appraised and 66 articles were included and the domains identified were theories; indications; types; structure; and benefits and challenges of group reflections.

CONCLUSIONS

Scaffolded by current approaches to individual reflections and theories and inculcated with nuanced adaptations from other medical practices, this SSR in SEBA suggests that structured group reflections may fill current gaps in training. However, design and assessment of the evidence-based structuring of group reflections proposed here must be the focus of future study.

Struggles and Joys: A Mixed Methods Study of the Artefacts and Reflections in Medical Student Portfolios

Introduction

Portfolios scaffold reflection on experience so students can plan their learning. To elicit reflection, the learning experiences documented in portfolios must be meaningful. To understand what experiences first- and second-year medical students find meaningful, we studied the patterns in the artefacts chosen for portfolios and their associated written reflections.

Methods

This explanatory mixed methods study of a longitudinal dataset of 835 artefacts from 37 medical student' portfolios, identified patterns in artefact types over time. Mixed model logistic regression analysis identified time, student and curriculum factors associated with inclusion of the most common types of artefacts. Thematic analysis of participants' reflections about their artefacts provided insight into their choices. Interpretation of the integrated findings was informed by Transformative Learning (TL) theory.

Results

Artefact choices changed over time, influenced by curriculum changes and personal factors. In first year, the most common types of artefacts were Problem Based Learning mechanism diagrams and group photos representing classwork; in second year written assignments and 'selfies' representing social and clinical activities. Themes in the written reflections were Landmarks and Progress, Struggles and Strategies, Connection and Collaboration, and Joyful Memories for Balance. Coursework artefacts and photographic self-portraits represented all levels of transformative learning from across the curriculum.

Conclusions

Medical students chose artefacts to represent challenging and/or landmark experiences, balanced by experiences that were joyful or fostered peer connection. Novelty influenced choice. To maximise learning students should draw from all experiences, to promote supported reflection with an advisor. Tasks should be timed to coincide with the introduction of new challenges.

Twelve tips for introducing E-Portfolios in health professions education

Portfolios have been used in health professions for many decades as a means of documenting reflective practice that inform change, supports the understanding of professionals' development needs and changing care options for clients. Electronic versions of one's portfolio of evidence or E-Portfolios became more prevalent in the early 2000s as a repository to store evidence and reflections. However, in recent years E-Portfolios have evolved from a repository to an articulation of authentic learning and development. Introduction of a range of E-Portfolio technology options, hosting systems and increasing professional/ethical standards, has resulted in challenges and opportunities for academics and professionals to meet increasing requirements for teaching and support of students training for health professions. This paper explores twelve tips explained and justified in a Health Science context, designed to support students developing and using E-Portfolios to fulfil professional standards, show evidence of reflective practice and culminate in securing a job in their field.

Portfolios in practice: Developing advancing practice within a musculoskeletal competency-based model

INTRODUCTION

The development of professional portfolios and the relevance of this within professional practice, competency and capability is gaining significant credibility in line with professional requirements. Nursing and medicine in terms of historical perspectives have long held the need for clinicians to maintain a portfolio for professional validation, whilst in other professional groups it is a requirement of registration. The allied health professionals, physiotherapy and ultimately musculoskeletal practice within this context are rapidly developing advancing and consultant practice. This professional development further requires appropriate verification and validation of practice, and achieving this can be through formal and non-formal routes.

PURPOSE

This paper looks to explore this and give direction to professionals developing portfolios whilst placing the requirements in context to contemporary practice in the U.K. Universities, professional bodies and special interest groups are now aligning in the need to support practice in a multi-format way, that moves away from traditional methods of evaluation into more diverse models of competency-based assessment.

IMPLICATIONS

With improvement in technology, the development of national frameworks and standards, portfolios in practice although commonly considered as standard practice will be a requirement not only of registration but as a criteria of maintaining status, career development and expansion of roles.

BACKGROUND

Musculoskeletal (MSK) physiotherapy in the U.K. has moved forward significantly in the last 20 years. Sitting within a clinical reasoned framework, the introduction of additional skills such is image requesting, injection therapies, and non-medical prescribing has further underpinned the advanced practice agenda (Langridge et al., 2015). While these advancements in practice are driving the profession forward, challenges remain in providing the workforce with a clear process of career development. Alongside developing professional pathways methods of evidencing advanced knowledge and skills acquired outside formal routes are required to support practitioners' career pathway into advancing practice.

Informed self-assessment during the transition to medical school: a longitudinal narrative study

OBJECTIVES

To explore how medical students' narratives of informed self-assessment (ISA) change during their first 18 months of study.

DESIGN

This longitudinal study used student narratives drawn from qualitative interviews and written reflections during the transition to medical school, to examine changes in ISA. Our analysis was informed by Situated Cognition Theory which recognises the impact and interplay of personal and environmental factors in cognition.

SETTING

To study medicine, first year students need to adapt their self-regulated learning in the context of a new peer group, study demands and educational culture. During this adaptation, students need to seek and interpret available cues to inform their self-assessment.

PARTICIPANTS

Longitudinal data were collected at five time points over 18 months from a diverse sample of seven first year medical students in an undergraduate medical programme, including 13.5 hours of interviews and 12 written reflections.

RESULTS

Before and after starting medical school, the participants' self-assessments were informed by environmental influences (exam results and comparison with peers), and personal influences (fear of failure and anxiety about not belonging). Early uncertainty meant self-assessments were overestimated and underestimated.By the end of first year, an enhanced sense of belonging coincided with less fear of failure, less emphasis on exam performance and reduced competition with peers. Self-assessments became increasingly informed by evidence of clinical skills and knowledge gained related to future professional competence.

CONCLUSION

Influences on medical students' ISAs change during the transition to studying medicine. A greater sense of belonging, and evidence of progress towards clinical competence became more important to self-assessment than comparison with peers and exam performance. Our findings reinforce the importance of formative assessments, opportunities to study and socialise with peers and early clinical experiences during first year. These experiences enhance ISA skills during the transition to medical school.

The ways social networks shape reflection on early significant clinical experiences in medical school

BACKGROUND

Medical curricula are increasingly providing opportunities to guide reflection for medical students. However, educational approaches are often limited to formalized classroom initiatives where reflection is prescriptive and measurable. There is paucity of literature that explores the personal ways students may experience authentic reflection outside of curricular time. The purpose of this study was to understand how social networks might shape dimensions of reflection.

METHODS

This study employed a qualitative social network analysis approach with a core sample of seven first year undergraduate medical students who described their relationships with 61 individuals in their networks. Data consisted of participant generated sociograms and individual semi-structured interviews.

RESULTS

Many learners struggled to find significant ways to involve their social networks outside of medicine in their new educational experiences. It appeared that some medical students began in-grouping, becoming more socially exclusive. Interestingly, participants emphasized how curricular opportunities such as reflective portfolio sessions were useful for capturing a diversity of perspectives.

CONCLUSIONS

Our study is one of the first to characterize the social networks inside and outside of medical school that students utilize to discuss and reflect on early significant clinical experiences. Recent commentary in the literature has suggested reflection is diverse and personal in nature and our study offers empirical evidence to demonstrate this. Our insights emphasize the importance of moving from an instrumental approach to an authentic socially situated approach if we wish to cultivate reflective lifelong learning.

Technology enhanced assessment: Ottawa consensus statement and recommendations

INTRODUCTION

In 2011, a consensus report was produced on technology-enhanced assessment (TEA), its good practices, and future perspectives. Since then, technological advances have enabled innovative practices and tools that have revolutionised how learners are assessed. In this updated consensus, we bring together the potential of technology and the ultimate goals of assessment on learner attainment, faculty development, and improved healthcare practices.

METHODS

As a material for the report, we used the scholarly publications on TEA in both HPE and general higher education, feedback from 2020 Ottawa Conference workshops, and scholarly publications on assessment technology practices during the Covid-19 pandemic.

RESULTS AND CONCLUSION

The group identified areas of consensus that remained to be resolved and issues that arose in the evolution of TEA. We adopted a three-stage approach (readiness to adopt technology, application of assessment technology, and evaluation/dissemination). The application stage adopted an assessment 'lifecycle' approach and targeted five key foci: (1) Advancing authenticity of assessment, (2) Engaging learners with assessment, (3) Enhancing design and scheduling, (4) Optimising assessment delivery and recording learner achievement, and (5) Tracking learner progress and faculty activity and thereby supporting longitudinal learning and continuous assessment.

The role of ePortfolios in supporting learning in eight healthcare disciplines: A scoping review

AIM

This scoping review aims at providing a summary of available knowledge about the role of ePortfolios in scaffolding learning in eight healthcare disciplines to identify main concepts, best practices, and knowledge gaps.

BACKGROUND

ePortfolios are well established in scaffolding learning in many healthcare disciplines. Yet, an overview of the ePortfolio literature in the context of healthcare education doesn't exist.

DESIGN AND METHODS

A scoping review of ePortfolio use in the following healthcare disciplines was conducted: audiology, dental hygiene, midwifery, nursing (associate degree and bachelor), occupational therapy, podiatry, and speech therapy. Eight databases were systematically searched. By structuring the data, five themes were identified: (1) contexts, (2) research populations and samples, (3) ePortfolio platforms used and objectives, (4) benefits and challenges in the use of ePortfolios, and (5) recommendations derived from literature in the use of ePortfolios. The themes were further specified with sub-themes. The themes help meeting the aim of the scoping review to develop a structured overview of ePortfolio use and research.

RESULTS

Thirty-seven articles were included. Samples predominantly involved students and fewer mentors, educators, and graduates. ePortfolios were used to support learning in the context of healthcare workplaces (n = 17), educational institutions (n = 14), or both (n = 6). Different objectives of ePortfolio use were derived from literature and categorized into eight themes: (1) to document and evidence competency, (2) reflection, (3) feedback and assessment, (4) (inter)professional collaboration, (5) support Continuous Professional Development, (6) bridging the gap between theory and practice, (7) employment, and (8) certification. These objectives were in turn coupled to two identified types of ePortfolio platforms: "store and communicate" and "work and learn" ePortfolios. The description of benefits and challenges formed a substantial part of the literature and data were mapped into five themes to structure the data: (1) technology, (2) collaboration, (3) competency development (before and after graduation), (4) employment, and (5) quality of patient care. Although the benefits outweighed the challenges, important challenges were identified, e.g., the high time investment required when working with an ePortfolio, low digital literacy, lack of internet access.

CONCLUSIONS

Studies mainly focused on perceptions (n = 32). Only a few studies explored measurable outcomes (i.e., competencies or behavioral indicators). Because ePortfolios are used by different ePortfolio users, in different contexts, and with different objectives, new research directions are needed to investigate how specific features, such as collaboration tools, feedback tools, or assessment tools can improve the role of ePortfolios in supporting healthcare education.

TWEETABLE ABSTRACT

This scoping review presents an overview of the knowledge about the role of ePortfolios in scaffolding learning in eight healthcare disciplines.

Taking the Big Leap: A Case Study on Implementing Programmatic Assessment in an Undergraduate Medical Program

The concept of programmatic assessment (PA) is well described in the literature; however, studies on implementing and operationalizing this systemic assessment approach are lacking. The present case study developed a local instantiation of PA, referred to as Assessment System Fribourg (ASF), which was inspired by an existing program. ASF was utilized for a new competency-based undergraduate Master of Medicine program at the State University of Fribourg. ASF relies on the interplay of four key principles and nine main program elements based on concepts of PA, formative assessment, and evaluative judgment. We started our journey in 2019 with the first cohort of 40 students who graduated in 2022. This paper describes our journey implementing ASF, including the enabling factors and hindrances that we encountered, and reflects on our experience and the path that is still in front of us. This case illustrates one possibility for implementing PA.

Embedding a Coaching Culture into Programmatic Assessment

Educational change in higher education is challenging and complex, requiring engagement with a multitude of perspectives and contextual factors. In this paper, we present a case study based on our experiences of enacting a fundamental educational change in a medical program; namely, the steps taken in the transition to programmatic assessment. Specifically, we reflect on the successes and failures in embedding a coaching culture into programmatic assessment. To do this, we refer to the principles of programmatic assessment as they apply to this case and conclude with some key lessons that we have learnt from engaging in this change process. Fostering a culture of programmatic assessment that supports learners to thrive through coaching has required compromise and adaptability, particularly in light of the changes to teaching and learning necessitated by the global pandemic. We continue to inculcate this culture and enact the principles of programmatic assessment with a focus on continuous quality improvement.

Everybody is able to reflect, or aren't they? Evaluating the development of medical professionalism via a longitudinal portfolio mentoring program from a student perspective

Introduction: Reflective competence is fundamental for responsible medical practice and must be systematically incorporated in medical training. To promote this, a longitudinal portfolio-based mentoring program was made mandatory at the Medical Faculty of the University of Tübingen in 2013. This study examines medical students' attitudes toward professional reflection and toward the program in general to draw conclusions about conditions as well as the needs-based design of the program. Method: In winter semester 2017/18, a retrospective questionnaire survey with free text fields was conducted (total sample: N=1.405; students S 1-9; response 37%; S 1-4 "Pre-clinic": n=231; S 5-9 "Clinic": n=241). Opinion trends of semester groups were identified through seven semi-structured interviews with semester speaker and peer tutors. Results: Differences in understanding and attitudes resulted in three positions: 1=approval, 2=ambivalence, 3=rejection. All three groups included individuals from pre-clinical and clinical settings with varying levels of experience. Prior experience and hidden curriculum influenced the position. Opinion trends confirmed the feedback. Conclusion: Although reflection appears in the National Competence-based Learning Objectives Catalogue for Medicine (NKLM), reflective competence is not regarded as equivalent to other study content. Motivation, commitment on the part of the mentors, and a trusting mentor-mentee relationship are effective. The flexibility of the portfolio in terms of content and methodology, as well as the curricular integration of the program are also beneficial.

Facilitators and Barriers of Reflective Learning in Postgraduate Medical Education: A Narrative Review

BACKGROUND

Reflection in postgraduate medical education has been found to aid in the development of professional skills, improve clinical expertise, and problem solving with the aim of advancing lifelong learning skills and self-awareness, leading to good medical practice among postgraduate residents. Despite the evidenced benefits, reflection remains underused as a tool for teaching and learning, and few trainee physicians regularly engage in the process. Factors that affect the uptake of reflective learning in residency training have not yet been adequately explored.

OBJECTIVE

The purpose of this review is to demonstrate the factors that influence the adoption of reflective learning for postgraduate students and their centrality to good clinical practice.

METHODS

A review of the literature was performed using defined databases and the following search terms: 'reflection', 'reflective learning', 'postgraduate medical education', 'barriers' and 'facilitators'. The search was limited to peer-reviewed published material in English between 2011 and 2020 and included research papers, reviews, and expert opinion pieces.

RESULTS

Eleven relevant articles were included, which identified three main categories as facilitators and barriers to the adoption of reflective learning in postgraduate medical education. These included structure, assessment and relational factors. The structure of reflective practice is important, but it should not be too rigid. Assessments are paramount, but they should be multidimensional to accommodate the multicomponent nature of reflections. Relational factors such as motivation, coaching, and role modeling facilitate sustainable reflective practice.

CONCLUSIONS

This review suggests that the same factors that facilitate reflection can be a barrier if not used within the right epistemic. Educators should consider these factors to increase the acceptance and integration of reflective learning in curriculums by both teachers and learners.

Improving Self-Reflection Assessment Practices: Comparative Judgment as an Alternative to Rubrics

CONSTRUCT

The authors aimed to investigate the utility of the comparative judgment method for assessing students' written self-reflections.

BACKGROUND

Medical practitioners' reflective skills are increasingly considered important and therefore included in the medical education curriculum. However, assessing students' reflective skills using rubrics does not appear to guarantee adequate inter-rater reliabilities. Recently, comparative judgment was introduced as a new method to evaluate performance assessments. This study investigates the merits and limitations of the comparative judgment method for assessing students' written self-reflections. More specifically, it examines the reliability in relation to the time spent assessing, the correlation between the scores obtained using the two methods (rubrics and comparative judgment), and, raters' perceptions of the comparative judgment method.

APPROACH

Twenty-two self-reflections, that had previously been scored using a rubric, were assessed by a group of eight raters using comparative judgment. Two hundred comparisons were completed and a rank order was calculated. Raters' impressions were investigated using a focus group.

FINDINGS

Using comparative judgment, each self-reflection needed to be compared seven times with another self-reflection to reach a scale separation reliability of .55. The inter-rater reliability of rating (ICC, (1, k)) using rubrics was .56. The time investment required for these reliability levels in both methods was around 24 minutes. The Kendall's tau rank correlation indicated a strong correlation between the scores obtained via both methods. Raters reported that making comparisons made them evaluate the quality of self-reflections in a more nuanced way. Time investment was, however, considered heavy, especially for the first comparisons. Although raters appreciated that they did not have to assign a grade to each self-reflection, the fact that the method does not automatically lead to a grade or feedback was considered a downside.

CONCLUSIONS

First evidence was provided for the comparative judgment method as an alternative to using rubrics for assessing students' written self-reflections. Before comparative judgment can be implemented for summative assessment, more research is needed on the time investment required to ensure no contradictory feedback is given back to students. Moreover, as the comparative judgment method requires an additional standard setting exercise to obtain grades, more research is warranted on the merits and limitations of this method when a pass/fail approach is used.

Reading the Self: Medical Students' Experience of Reflecting on Their Writing Over Time

PURPOSE

To investigate students' experience (over time) with meta-reflection writing exercises, called Signature Reflections. These exercises were used to strengthen reflective capacity, as part of a 4-year reflective writing portfolio curriculum that builds on a recognized strategy for reflection (narrative medicine) and employs longitudinal faculty-mentors.

METHOD

In 2018, the authors conducted 5 focus groups with 18 third-year students from the Columbia University Vagelos College of Physicians and Surgeons class of 2019 to examine students' experience with Signature Reflections. Using an iterative, thematic approach, they developed codes to reflect common patterns in the transcripts, distilled conceptually similar codes, and assembled the code categories into themes.

RESULTS

Three core themes (safe space, narrative experience, mirror of self) and 1 overarching theme (moving through time) were identified. Students frequently experienced relief at having a safe reflective space that promoted grappling with their fears or vulnerabilities and highlighted contextual factors (e.g., trusted faculty-mentors, protected time) that fostered a safe space for reflection and exploration. They often emphasized the value of tangible documentation of their medical school journey (narrative experience) and reported using Signature Reflections to examine their emerging identity (mirror of self). Overlapping with the core themes was a deep appreciation for the temporal perspective facilitated by the Signature Reflections (moving through time).

CONCLUSIONS

A longitudinal narrative medicine-based portfolio curriculum with pauses for meta-reflection allowed students, with faculty support, to observe their trajectory through medical school, explore fears and vulnerabilities, and narrate their own growth. Findings suggest that narrative medicine curricula should be required and sufficiently longitudinal to facilitate opportunities to practice the skill of writing for insight, foster relationships with faculty, and strengthen students' temporal perspectives of their development.

Can concept mapping support the quality of reflections made by undergraduate medical students? A mixed method study

BACKGROUND

Students perceive reflective writing as difficult. Concept mapping may be an alternative format for reflection, which provides support while allowing students to freely shape their thoughts. We examined (1) the quality of reflection in reflective concept maps created by first-year medical students and (2) students' perceptions about concept mapping as a tool for reflection.

METHODS

Mixed-method study conducted within the medical curriculum of Maastricht University, The Netherlands, consisting of: (1) Analysis of the quality of reflection in 245 reflective concept maps created by 40 first-year students. Reflection quality was analysed by assessing focus of reflection (technical/practical/sensitising) and depth of reflection (description/justification/critique/discussion). (2) Semi-structured interviews with 22 students to explore perceived effectiveness of reflective concept mapping.

RESULTS

Depth of reflection reached at least the level of critique in 82% of maps. Three factors appeared to affect the perceived effectiveness of concept mapping for reflection: (1) reflective concept map structure; (2) alertness to meaningful experiences in practice and (3) learning by doing.

CONCLUSION

These results yielded supportive evidence for concept mapping as a useful technique to teach novice learners the basics of effective reflection. Meaningful implementation requires a delicate balance between providing a supportive structure and allowing flexibility for the student.

Development of a portfolio framework for implementation of an outcomes-based healthcare professional education curriculum using a modified e-Delphi method

Background

The explicit declaration of Entrustable Professional Activities (EPA) and milestones are an essential component of a competency based medical education curricula. The present study attempts to develop a portfolio framework to document them for adaptation in any healthcare professional education curriculum development.

Methods

A modified e-Delphi method was used after incorporation of a study group of medical education experts (MEDEX-G). Consensus was defined as 75% agreement. Both qualitative and quantitative data was collected and analysed to conduct three rounds of the Delphi.

Results

The draft template was prepared by the core faculty of medical education centre. The final template was approved by the experts after 03 iterations of anonymous online voting and presentation of summary results by the moderator. The final template lists out the milestones of each EPA separately with provision for expected expertise and level, suggestive teaching learning activities and assessments, reflections by the students and feedback by facilitator for each EPA.

Conclusion

The MEDEX-G consensus template is an important development to encourage the adoption of portfolios in a competency based medical education curriculum. It can be freely adopted by various healthcare professional education bodies in various disciplines.

E-portfolios and Entrustable Professional Activities to support competency-based education in dietetics

The concept of Entrustable Professional Activities, recently pioneered in medical education, has emerged to support the implementation of competency-based education. Although competency-based frameworks are widely used in healthcare professional education to develop outcomes-based curricula, assessment of student competency in professional placement settings remains challenging. The novel concept of Entrustable Professional Activities together with established methods of competency assessment, namely e-portfolios and self-assessment, was implemented in the "[La Trobe University Dietetic program in 2015-2016. This study aimed to appraise the e-portfolio and evaluate the use of Entrustable Professional Activities to assess competence. A mixed-methods evaluation, using qualitative and quantitative surveys with follow-up structured consultations, was conducted with final year dietetics students and their supervisors. Dietetics students were comfortable with Entrustable Professional Activities and competency-based assessment, whereas supervisors preferred Entrustable Professional Activity based assessment. All stakeholders valued student self-assessment and the ongoing use of structured e-portfolios to develop and document competency. The use of structured e-portfolios, student self-assessment, and the emerging concept of Entrustable Professional Activities are useful tools to support dietetics student education in professional placement settings.

How is self-regulated learning documented in e-portfolios of trainees? A content analysis

BACKGROUND

It is assumed that portfolios contribute to self-regulated learning (SRL). Presence of these SRL processes within the documentation kept in portfolios is presupposed in common educational practices, such as the assessment of reflective entries. However, questions can be asked considering the presence of SRL within portfolios. The aim of this study was to gain insight into the documentation of SRL processes within the electronic (e)-portfolio content of medical trainees. SRL consists of numerous processes, for this study the focus was on self-assessment via reflection and feedback, goal-setting and planning, and monitoring, as these are the processes that health professions education research mentions to be supported by portfolios.

METHODS

A database containing 1022 anonymous e-portfolios from General Practitioner trainees was used to provide descriptive statistics of the various available e-portfolio forms. This was followed by a quantitative content analysis of 90 e-portfolios, for which, a codebook was constructed to rate the documentation of the included SRL processes.

RESULTS

The numbers of forms in the e-portfolios varied to a great extent. Content analysis showed a limited documentation of reflective entries, and available entries mainly described events and experiences without explanations and context. Feedback was generally limited to comments on what went well and lacked specificity, context and suggestions for future action. Learning goals and plans were short of specificity, but did contain challenging topics and different goals were compatible with each other. 75% of the e-portfolios showed (limited) signs of monitoring.

CONCLUSIONS

The e-portfolio content showed limited documentation of SRL processes. As documentation of SRL requires time and asks for a high level of introspection and writing skills, one cannot expect documentation of SRL processes to appear in e-portfolio content without efforts.

Student perspectives on competency-based portfolios: Does a portfolio reflect their competence development?

INTRODUCTION

Portfolio-based assessments require that learners' competence development is adequately reflected in portfolio documentation. This study explored how students select and document performance data in their portfolios and how they perceive these data to be representative for their competence development.

METHODS

Students uploaded performance data in a competency-based portfolio. During one clerkship period, twelve students also recorded an audio diary in which they reflected on experiences and feedback that they perceived to be indicants of their competence development. Afterwards, these students were interviewed to explore the extent to which the performance documentation in the portfolio corresponded with what they considered illustrative evidence of their development. The interviews were analyzed using thematic analysis.

RESULTS

Portfolios provide an accurate but fragmented picture of student development. Portfolio documentation was influenced by tensions between learning and assessment, student beliefs about the goal of portfolios, student performance evaluation strategies, the learning environment and portfolio structure.

DISCUSSION

This study confirms the importance of taking student perceptions into account when implementing a competency-based portfolio. Students would benefit from coaching on how to select meaningful experiences and performance data for documentation in their portfolios. Flexibility in portfolio structure and requirements is essential to ensure optimal fit between students' experienced competence development and portfolio content.

Nationwide introduction of a new competency framework for undergraduate medical curricula: a collaborative approach

Switzerland recently introduced PROFILES, a revised version of its national outcomes reference framework for the undergraduate medical curriculum. PROFILES is based on a set of competencies adapted from the CanMEDS framework and nine entrustable professional activities (EPAs) that students have to be able to perform autonomously in the context of a predefined list of clinical situations. The nationwide implementation of such a competency- and EPA-based approach to medical education is a complex process that represents an important change to the organisation of undergraduate training in the various medical schools. At the same time, the concepts underlying PROFILES also have to be reflected at the level of the Federal Licencing Examination (FLE) and the national accreditation process. The vice-deans for education mandated a Swiss Working Group for PROFILES Implementation (SWGPI) to elaborate a guide presenting the principles and best practices based on the current scientific literature, to ensure the coherence between the future developments of the medical curricula and the evolution of the FLE, and to propose a coordinated research agenda to evaluate the implementation process. On the basis of the literature and analysis of our national context, we determined the key elements important for a successful implementation. They can be grouped into several areas including curricular design and governance, the assessment system and entrustment process, faculty development and change management. We also identified two dimensions that will be of particular importance to create synergies and facilitate exchange between the medical schools: a systematic approach to curriculum mapping and the longitudinal integration of an e-portfolio to support the student learning process. The nationwide collaborative approach to define strategies and conditions for the implementation of a new reference framework has allowed to develop a shared understanding of the implications of PROFILES, to promote the establishment of Swiss mapping and e-portfolio communities, and to establish the conditions necessary for ensuring the continuous alignment of the FLE with the evolving medical curricula.

Use of portfolios in teaching communication skills and professionalism for Portuguese-speaking medical students

OBJECTIVES

This study aimed to analyse the effect of a portfolio with three activities fostering students' reflection, self-efficacy and teaching of communication skills and professionalism.

METHODS

A cross-sectional study was applied with a sample of third- and fourth-year medical students in one Portuguese and three Brazilian universities. A three-activity portfolio (course evaluation and learning, self-efficacy activity and free reflective writing) was used during a two-month course on communication skills and professionalism. The 69 students enrolled in the course were invited to complete the three-activity portfolio via Likert-type questionnaires, open-ended questions and narrative. Content and lexical analysis and the Reflection Evaluation for Learners' Enhanced Competencies Tool (REFLECT) were used for assessing the qualitative data. The questionnaires were evaluated using principal components analysis and Cronbach's α. Pearson's correlation was applied to portfolio activities.

RESULTS

Of the 69 participants, 85.5% completed at least one activity. Reflecting on what they learned in the communication module, the students did not mention professionalism themes. In the self-efficacy activity on communication, 25% of the fragments were related to professionalism themes. There was a negative correlation between students' self-efficacy and the REFLECT rubric score (r₍₁₉₎=-0.744; p< 0.0001).

CONCLUSIONS

Teachers must consider the activity's influence on the reflections when assessing the portfolio. This model of a three-activity portfolio provided diverse ways of encouraging and assessing reflections, supporting teaching improvement and adaptation, evaluating students' self-efficacy and showing that students' higher reflective capacity may promote feelings of low effectiveness.

Experiences of recent graduates: reframing reflection as purposeful, social activity

BACKGROUND

During the Royal College of Veterinary Surgeons' (RCVS) Professional Development Phase, graduates are required to reflect on their progress. Reflection is often conceptualised as a solitary activity, which may contrast with day-to-day reflective activities in the workplace. This study drew on cultural-historical activity theory to understand how recently graduated veterinary surgeons engage in reflective activity.

METHODS

Data comprised RCVS documentation and semistructured interviews with 15 recent graduates from one veterinary school. Thematic analysis was used to describe a collective system of reflective activity and to identify contradictions in the system with the potential to limit outcomes of reflective activity.

RESULTS

Two overarching themes of contradictions were identified: 'social reflection' and 'formalising the informal'. Graduates need opportunities for talking and/or writing to progress worries into purposeful reflection, underpinned by a shared understanding of reflective activity with colleagues, and by working practices which prioritise and normalise reflective interaction.

CONCLUSION

These findings identify potential avenues to better support veterinary graduates as they negotiate the transition to working life, and suggest that reconsideration of the formal expectations of new veterinary graduates and their employers is timely.

Multiple-role mentoring: mentors' conceptualisations, enactments and role conflicts

INTRODUCTION

Outcome-based approaches to education and the inherent emphasis on programmatic assessment in particular, require models of mentoring in which mentors fulfil dual roles: coach and assessor. Fulfilling multiple roles could result in role confusion or even role conflicts, both of which may affect mentoring processes and outcomes. In this study, we explored how mentors conceptualise and enact their role in a multiple-role mentoring system and to what extent they experience role conflicts.

METHODS

We conducted a constructivist grounded theory study at one undergraduate medical school. A purposive sample of 12 physician-mentors active in a programmatic assessment system was interviewed. Data analysis followed stages of open, axial and selective coding through which themes were constructed.

RESULTS

Three predominant mentoring approaches were constructed: (i) empowering (a reflective and holistic approach to student development); (ii) checking (an observant approach to check whether formal requirements are met), and (iii) directing (an authoritative approach to guide students' professional development). Each approach encompassed a corresponding type of mentor-mentee relationship: (i) partnership; (ii) instrumental, and (iii) faculty-centred. Furthermore, mentors' strategies, focus, agency provided to students and perception of the assessment system characterised mentoring approaches and relationships. Role conflicts were mainly experienced by mentors with a directing mentoring approach. They used various coping mechanisms, including deviation from assessment guidelines.

CONCLUSIONS

In multiple-role mentoring in the context of programmatic assessment, mentors adopted certain predominant mentoring approaches, which were characterised by different strategies for mentoring and resulted in different mentor-mentee relationships. Multiple-role mentoring does not necessarily result in role conflict. Mentors who do experience role conflict seem to favour the directing approach, which is most at odds with key principles of competency-based education and programmatic assessment. These findings build upon existing mentoring literature and offer practical suggestions for faculty development regarding approaches to mentoring in programmatic assessment systems.

Inter-Rater Reliability of Grading Undergraduate Portfolios in Veterinary Medical Education

The reliability of high-stakes assessment of portfolios containing an aggregation of quantitative and qualitative data based on programmatic assessment is under debate, especially when multiple assessors are involved. In this study carried out at the Faculty of Veterinary Medicine, Utrecht University, the Netherlands, two independent assessors graded the portfolios of students in their second year of the 3-year clinical phase. The similarity of grades (i.e., equal grades) and the level of the grades were studied to estimate inter-rater reliability, taking into account the potential effects of the assessor's background (i.e., originating from a clinical or non-clinical department) and student's cohort group, gender, and chosen master track (Companion Animal Health, Equine Health, or Farm Animal/Public Health). Whereas the similarity between the two grades increased from 58% in the first year the grading system was introduced to around 80% afterwards, the grade level was lower over the next 3 years. The assessor's background had a minor effect on the proportion of similar grades, as well as on grading level. The assessor intraclass correlation was low (i.e., all assessors scored with a similar grading pattern [same range of grades]). The grades awarded to female students were higher but more often dissimilar. We conclude that the grading system was well implemented and has a high inter-rater reliability.

Judging residents' performance: a qualitative study using grounded theory

BACKGROUND

Although program directors judge residents' performance for summative decisions, little is known about how they do this. This study examined what information program directors use and how they value this information in making a judgment of residents' performance and what residents think of this process.

METHODS

Sixteen semi-structured interviews were held with residents and program directors from different hospitals in the Netherlands in 2015-2016. Participants were recruited from internal medicine, surgery and radiology. Transcripts were analysed using grounded theory methodology. Concepts and themes were identified by iterative constant comparison.

RESULTS

When approaching semi-annual meetings with residents, program directors report primarily gathering information from the following: assessment tools, faculty members and from their own experience with residents. They put more value on faculty's comments during meetings and in the corridors than on feedback provided in the assessment tools. They are influenced by their own beliefs about learning and education in valuing feedback. Residents are aware that faculty members discuss their performance in meetings, but they believe the assessment tools provide the most important proof to demonstrate their clinical competency.

CONCLUSIONS

Residents think that feedback in the assessment tools is the most important proof to demonstrate their performance, whereas program directors scarcely use this feedback to form a judgment about residents' performance. They rely heavily on remarks of faculty in meetings instead. Therefore, residents' performance may be better judged in group meetings that are organised to enhance optimal information sharing and decision making about residents' performance.

The reflective zombie: Problematizing the conceptual framework of reflection in medical education

Reflection is an ambiguous and profoundly complex human activity. We celebrate the developments in teaching and researching reflection in education, yet have identified flaws in the way reflection has been operationalized: medical education has translated the age-old concept into a teachable and measureable construct. We fear that in this process of operationalization, the philosophical underpinnings of reflection have been discarded. We illustrate this with a thought experiment about a 'reflective zombie': students who have been conditioned to follow prescribed thought steps rather than engaging in truly reflective behaviour. In research and assessment of reflection, measuring tools might be unable to distinguish reflective zombies from students who authentically reflect. We argue that the instrumental approach lies at the root of this problem as it limits the rich concept of reflection and illustrate our point by describing problems related to paradigm (we are looking at reflection in the wrong way), methods (we are using the wrong tools), and epistemics (can we even know what we want to know?). We offer three suggestions for implementing reflection into the curriculum and for research into reflection. First, acknowledge the diversity of reflection and let go of the 'checklist approach'. Second, embrace the personal nature of reflection by stimulating awareness of one's personal reflection styles as part of the reflective process. Third, shift the focus of research to the practice of reflection. We believe that a strong vision on reflection can lead to a balanced curriculum, setting students up for a lifelong learning as a reflective practitioner.

Faculty Development Revisited: A Systems-Based View of Stakeholder Development to Meet the Demands of Entrustable Professional Activity Implementation

In 2015, the Association of American Medical Colleges implemented an interinstitutional pilot of 13 core entrustable professional activities (EPAs) for entering residency, activities that entering residents should be expected to perform with indirect supervision. The pilot included a concept group on faculty development; this group previously offered a shared mental model focused on the development of faculty who devote their efforts to clinical teaching and assessment for learning and entrustment decision making. In this article, the authors draw from the literature of competency-based education to propose what is needed in overall approaches to faculty development to prepare institutions for undergraduate EPA implementation.Taking a systems-based view that defines the necessary tasks of EPA implementation, the authors move beyond the variably used term "faculty" and enumerate a comprehensive list of institutional stakeholders who can meaningfully support and/or engage in the relationships and organizational processes required for EPA learning and assessment. They consider each group's responsibilities and development needs according to five domains delineated by Steinert: teaching improvement, leadership and management, research-building capacity, academic career building, and organizational change.The authors argue that the EPA framework addresses barriers posed with the use of a competency-based framework. By facilitating the communication required for organizational change, enabling valid assessment with comprehensive yet feasible levels of faculty development, and incorporating all relevant data on student professional behavior into summative assessment decisions, EPAs may offer a clearer path toward the goal of competency-based education.

Twelve tips to promote successful development of a learner performance dashboard within a medical education program

Easily accessible and interpretable performance data constitute critical feedback for learners that facilitate informed self-assessment and learning planning. To provide this feedback, there has been a proliferation of educational dashboards in recent years. An educational (learner) dashboard systematically delivers timely and continuous feedback on performance and can provide easily visualized and interpreted performance data. In this paper, we provide practical tips for developing a functional, user-friendly individual learner performance dashboard and literature review of dashboard development, assessment theory, and users' perspectives. Considering key design principles and maximizing current technological advances in data visualization techniques can increase dashboard utility and enhance the user experience. By bridging current technology with assessment strategies that support learning, educators can continue to improve the field of learning analytics and design of information management tools such as dashboards in support of improved learning outcomes.

Looking Back to Move Forward: First-Year Medical Students' Meta-Reflections on Their Narrative Portfolio Writings

The day-to-day rigors of medical education often preclude learners from gaining a longitudinal perspective on who they are becoming. Furthermore, the current focus on competencies, coupled with concerning rates of trainee burnout and a decline in empathy, have fueled the search for pedagogic tools to foster students' reflective capacity. In response, many scholars have looked to the tradition of narrative medicine to foster "reflective spaces" wherein holistic professional identity construction can be supported. This article focuses on the rationale, content, and early analysis of the reflective space created by the narrative medicine-centered portfolio at the Columbia University Vagelos College of Physicians and Surgeons. In January 2015, the authors investigated learning outcomes derived from students' "Signature Reflections," end-of-semester meta-reflections on their previous portfolio work. The authors analyzed the Signature Reflections of 97 (of 132) first-year medical students using a constant comparative process. This iterative approach allowed researchers to identify themes within students' writings and interpret the data. The authors identified two overarching interpretive themes-recognition and grappling-and six subthemes. Recognition included comments about self-awareness and empathy. Grappling encompassed the subthemes of internal change, dichotomies, wonder and questioning, and anxiety. Based on the authors' analyses, the Signature Reflection seems to provide a structured framework that encourages students' reflective capacity and the construction of holistic professional identity. Other medical educators may adopt meta-reflection, within the reflective space of a writing portfolio, to encourage students' acquisition of a longitudinal perspective on who they are becoming and how they are constructing their professional identity.

Untying the Gordian knot: remediation problems in medical schools that need remediation

This position paper discusses on-going academic remediation challenges within the field of medical education. More specifically, we identify three common contemporary problems and propose four recommendations to strengthen remediation efforts. Selecting or determining what type of remediation is needed for a particular student is akin to analyzing a Gordian knot with individual, institutional and systemic contributors. More emphasis, including multi-institutional projects and research funding is needed. Recommendations regarding language use and marketing of such programs are given.

From aggregation to interpretation: how assessors judge complex data in a competency-based portfolio

While portfolios are increasingly used to assess competence, the validity of such portfolio-based assessments has hitherto remained unconfirmed. The purpose of the present research is therefore to further our understanding of how assessors form judgments when interpreting the complex data included in a competency-based portfolio. Eighteen assessors appraised one of three competency-based mock portfolios while thinking aloud, before taking part in semi-structured interviews. A thematic analysis of the think-aloud protocols and interviews revealed that assessors reached judgments through a 3-phase cyclical cognitive process of acquiring, organizing, and integrating evidence. Upon conclusion of the first cycle, assessors reviewed the remaining portfolio evidence to look for confirming or disconfirming evidence. Assessors were inclined to stick to their initial judgments even when confronted with seemingly disconfirming evidence. Although assessors reached similar final (pass-fail) judgments of students' professional competence, they differed in their information-processing approaches and the reasoning behind their judgments. Differences sprung from assessors' divergent assessment beliefs, performance theories, and inferences about the student. Assessment beliefs refer to assessors' opinions about what kind of evidence gives the most valuable and trustworthy information about the student's competence, whereas assessors' performance theories concern their conceptualizations of what constitutes professional competence and competent performance. Even when using the same pieces of information, assessors furthermore differed with respect to inferences about the student as a person as well as a (future) professional. Our findings support the notion that assessors' reasoning in judgment and decision-making varies and is guided by their mental models of performance assessment, potentially impacting feedback and the credibility of decisions. Our findings also lend further credence to the assertion that portfolios should be judged by multiple assessors who should, moreover, thoroughly substantiate their judgments. Finally, it is suggested that portfolios be designed in such a way that they facilitate the selection of and navigation through the portfolio evidence.

Factors Influencing Acceptability and Perceived Impacts of a Mandatory ePortfolio Implemented by an Occupational Therapy Regulatory Organization

INTRODUCTION

The use of ePortfolios has been implemented in several regulatory organizations to encourage clinicians' engagement in continuing professional development (CPD). However, their use has achieved mixed success, and multiple personal and contextual factors can influence their impacts on practice change. The aim of this study was to identify which factors influence the acceptability and perceived impacts of an ePortfolio implemented by an occupational therapy regulatory organization in one Canadian province.

METHOD

A cross-sectional online survey design was used. The survey was sent to registered occupational therapists in Quebec. Multiple regression analyses were conducted to identify factors influencing acceptability and outcomes: ease of use, satisfaction, impact on implementation of the CPD plan, and competence improvement.

RESULTS

The survey was fully completed by 546 participants. Factors significantly influencing the ePortfolio acceptability and perceived impacts were attitude toward and familiarity with the portfolio, confidence in reflective skills, engagement in the CPD plan, and desire for feedback. Time spent completing the ePortfolio and the fact of completing it in teams were negatively associated with the outcomes.

DISCUSSION

Shaping more favorable user attitudes, helping users recognize and experience the tool's benefits for their practice, and fostering confidence in their reflective skills are important factors that can be addressed to improve ePortfolio acceptability and outcomes. Contextual factors, such as time spent completing the ePortfolio and completing it in teams, seem to reflect greater difficulty with using the tool. Study findings can contribute to improving ePortfolio implementation in the CPD context.

The use of a portfolio in postgraduate medical education - reflect, assess and account, one for each or all in one?

Competency-based education has become central to the training and assessment of post-graduate medical trainees or residents [1]. In competency-based education, there is a strong focus on outcomes and professional performance. Typically, holistic tasks are used to train, practice and assess the defined outcomes or competencies. In residency training, these tasks are part of the day-to-day clinical practice. The performance of residents in the workplace needs to be captured and stored. A portfolio has been used as an instrument for storage and collection of workplace-based assessment and feedback in various countries, like the Netherlands and the United States. The collection of information in a portfolio can serve or be used for a variety of purposes. These are: The collection of work samples, assessment, feedback and evaluations in a portfolio enables the learner to look back, analyze and reflect. The content is used for assessment or making decisions about progress. And the portfolio is used as an instrument for quality assurance processes. In post-graduate medical education, these purposes can be combined but this is not always reported transparently. In this paper, we will discuss the different perspectives, how a portfolio can serve these three purposes and what are opportunities and challenges of combining multiple purposes.

Targeting Assessment for Learning within Pharmacy Education

Formative assessment is critical for deliberate improvement, development and growth. While not entirely synonymous, assessment for learning (AFL) is an approach using formative assessment to specifically improve students' learning. While using formative assessments, AFL can also have summative programmatic-assessment implications. For each learning assessment, summative and formative uses can be leveraged; it can scaffold (formative), foster students' growth (formative), and document students' development in a competency/standard (summative). For example, using a developmental portfolio with iterative reflective-writings (formative), PharmD students showed qualitative development in the "professionalism" competency (summative; ACPE Standard 4.4). (In parallel, this development in professionalism was confirmed quantitatively.) An AFL approach can complement other assessments; it can be integrated with other summative assessments into a multi-method assessment program, wherein developmental portfolio sections could be used for a few specific competencies. While AFL is not a one-size-fits-all silver bullet approach for programmatic assessment, it is one notable robust tool to employ.

Utilisation of an electronic portfolio to engage rehabilitation professionals in continuing professional development: results of a provincial survey

INTRODUCTION

ePortfolios are frequently used to support continuing professional development (CPD) of rehabilitation professionals. Though this tool is now widely implemented in many professions by regulatory organisations, very few studies have investigated the use and impact among rehabilitation professionals. Implementation of comprehensive ePortfolios that are centred on the needs of rehabilitation professionals requires documenting their level of use and perceived outcomes.

OBJECTIVES

The objectives were to describe how occupational therapists use a mandatory ePortfolio that has been recently implemented by a regulatory organisation in Quebec (Canada) and the perceived outcomes of this requirement on continuing professional development and practice change.

METHODS

An online survey was sent to all registered occupational therapists in Quebec using the ePortfolio. The survey content was developed based on a literature review and expert consultation. Results were analysed using descriptive statistics.

RESULTS

A total of 546 respondents completed the survey. Results show relatively high levels of ease and satisfaction with the tool, but a limited perception of the tool's impacts on the improvement of professional competencies and change in practices.

CONCLUSIONS

Occupational therapists reported that use of the ePortfolio supports their engagement in CPD but has limited impact on practice. Promotion of work-based learning, team use and mentor support could increase its meaningfulness for professionals. Implications for Rehabilitation To improve attitudes and beliefs about benefits related to portfolio use, rehabilitation practitioners need a very clear understanding of the purpose and usefulness of a portfolio in clinical practice. Most of the respondents saw the ePortfolio as helping them develop and implement a continuing professional development plan and reflect on the changes needed in their practice. Portfolio use in teams and productive reflection should be promoted in order to target shared objectives for continuous practice improvement. Rehabilitation professionals trained in portfolio use during their entry-level studies have a slightly more positive attitude towards portfolio use and impact of this use compared with than clinicians who have not had this training.