A model of engagement in reflective writing-based portfolios: Interactions between points of vulnerability and acts of adaptability

Uncovering Reflections of Preclinical Medical Students from Clinical Shadowing: A Thematic Analysis

PURPOSE

Clinical shadowing(CS) offers preclinical medical students the opportunity to observe certified physicians in practice. Analyzing the content of students' reflective writings(RWs) to understand core competencies they perceived during CS, as well as stages of their reflections, can offer valuable insights for course design and instructor guidance. This study aimed to address this matter by employing a thematic analysis of students' RWs to explore the learning outcomes derived from CS.

MATERIALS AND METHODS

The dataset of the thematic analysis comprised de-identified RWs from preclinical medical students who participated in the clinical shadowing program during the first semester of the 2019 academic year at National Taiwan University College of Medicine. Two researchers independently extracted and categorized pertinent content from the texts of RWs, aligning them with the six core competencies of ACGME and mapping them to the stages of Kolb's learning cycle. Additionally, correlation with shadowing specialties and fields was investigated.

RESULTS

In total, 155 RWs were analyzed. Patient care emerged as the most frequently reflected competency(30%), followed by systems-based practice(23%), and medical knowledge(21%). The shadowing specialties and fields had an impact on the core competencies students perceived. In terms of Kolb's learning stages, concrete experience(66%) predominated in RWs, while abstract conceptualization(19%) and reflective observation(14%) were less prevalent. For each competency, concrete experience was the prevailing stage, with active experimentation being the least dominant one. Students exhibited varying proportions of reflections at different stages for each core competency.

CONCLUSIONS

Clinical shadowing experiences enable preclinical medical students to explore required clinical core competencies, with emphases depending on the shadowing specialties and fields. Currently, most reflections are concentrated at Kolb's concrete experience stage. This study offers valuable insights for designing future CS programs and improving faculty development, aimed at helping students achieve deeper and more comprehensive reflective learning in relation to core competencies, and even professional identity.

Discoveries or doubts: a qualitative study of the transformative potential of portfolio meetings

To adapt to medical school, students need to change their approaches to learning and study. Transformative learning through critical reflection on disorienting learning experiences supports perspective change to direct new activity. We explored how portfolio meetings support changes in students' perspectives towards learning and study during the transition to studying medicine. This qualitative mixed methods study explored changes in medical students' perspectives before and after two portfolio meetings with a mentor during the first year of medicine. Adopting a constructionist approach, we analyzed interview transcripts and written reflections from a diverse sample of students using reflexive thematic analysis. The findings were integrated during analysis. Transformative learning theory was our interpretive lens. Our analysis revealed five themes. Two themes represented students' initial apprehension about portfolio meetings: The Disclosure Dilemma and A Question of Priorities. The theme "Seeing the Big Picture" described new perspectives from reflection on learning while preparing for meetings. Clarity from Dialogue described changed perspectives to learning and study after meetings. In the theme Dialogue Disappointment, meetings perceived as unhelpful led to persisting doubts about the value of portfolio meetings. Transformative learning was evident when students described new insights into their learning leading to goal-setting and new study strategies. When initial meetings were helpful, doubts about portfolio meetings were dispelled, enhancing student engagement in future meetings. Not all meetings were transformative, highlighting the importance of student and mentor preparation and training. Further research is needed to determine whether early portfolio experiences shape later engagement in clinical contexts.

Struggles and Joys: A Mixed Methods Study of the Artefacts and Reflections in Medical Student Portfolios

Introduction

Portfolios scaffold reflection on experience so students can plan their learning. To elicit reflection, the learning experiences documented in portfolios must be meaningful. To understand what experiences first- and second-year medical students find meaningful, we studied the patterns in the artefacts chosen for portfolios and their associated written reflections.

Methods

This explanatory mixed methods study of a longitudinal dataset of 835 artefacts from 37 medical student' portfolios, identified patterns in artefact types over time. Mixed model logistic regression analysis identified time, student and curriculum factors associated with inclusion of the most common types of artefacts. Thematic analysis of participants' reflections about their artefacts provided insight into their choices. Interpretation of the integrated findings was informed by Transformative Learning (TL) theory.

Results

Artefact choices changed over time, influenced by curriculum changes and personal factors. In first year, the most common types of artefacts were Problem Based Learning mechanism diagrams and group photos representing classwork; in second year written assignments and 'selfies' representing social and clinical activities. Themes in the written reflections were Landmarks and Progress, Struggles and Strategies, Connection and Collaboration, and Joyful Memories for Balance. Coursework artefacts and photographic self-portraits represented all levels of transformative learning from across the curriculum.

Conclusions

Medical students chose artefacts to represent challenging and/or landmark experiences, balanced by experiences that were joyful or fostered peer connection. Novelty influenced choice. To maximise learning students should draw from all experiences, to promote supported reflection with an advisor. Tasks should be timed to coincide with the introduction of new challenges.

Grading reflective essays: the construct validity and reliability of a newly developed Tool- GRE-9

BACKGROUND

The main objective of this study is to assess the construct validity and reliability of the Grading Reflective Essays-9 (GRE-9).

METHODS

This study took place in a major tertiary academic medical center in Beirut, Lebanon. 104 reflective essays written by years 1-3 residents in the department of Family Medicine at the American University of Beirut Medical Center (AUBMC) were graded by 2 trained raters who independently scored the essays using GRE-9. GRE-9 scores were then correlated with scores on communication skills OSCE stations and in-training examinations to investigate, respectively, convergent and divergent validity. One of the 2 raters scored the essays twice one month apart to assess the reliability of the GRE-9 using intra rater reliability and internal consistency.

RESULTS

There was a weak, non-significant correlation between GRE-9 score and In training examination (ITE) score (r = - .213, p = .395). There was a moderate, non-significant correlation between GRE-9 scores and the Objective structured clinical examination (OSCE) communication station scores (r = - .412 p = .162). The correlation coefficient between trails 1 and 2 was significant (r = .832, p = .000). Intra class correlation coefficient (ICC) analysis demonstrated almost perfect intra-rater agreement (0.819; 95% CI: 0.741-0.875) of the test ratings over time.

CONCLUSIONS

GRE-9, is a short, concise, easy-to-use reliable grading tool for reflective essays that has demonstrated moderate to substantial intra-rater reliability and evidence of divergent validity. The study found non-significant correlations between reflective writing scores OSCE communication scores demonstrating a lack of relationship between reflective writing and this measure of performance.

Evidence-guided approach to portfolio-guided teaching and assessing communications, ethics and professionalism for medical students and physicians: a systematic scoping review

OBJECTIVES

Guiding the development of longitudinal competencies in communication, ethics and professionalism underlines the role of portfolios to capture and evaluate the multiple multisource appraisals and direct personalised support to clinicians. However, a common approach to these combined portfolios continues to elude medical practice. A systematic scoping review is proposed to map portfolio use in training and assessments of ethics, communication and professionalism competencies particularly in its inculcation of new values, beliefs and principles changes attitudes, thinking and practice while nurturing professional identity formation. It is posited that effective structuring of portfolios can promote self-directed learning, personalised assessment and appropriate support of professional identity formation.

DESIGN

Krishna's Systematic Evidence-Based Approach (SEBA) is employed to guide this systematic scoping review of portfolio use in communication, ethics and professionalism training and assessment.

DATA SOURCES

PubMed, Embase, PsycINFO, ERIC, Scopus and Google Scholar databases.

ELIGIBILITY CRITERIA

Articles published between 1 January 2000 and 31 December 2020 were included.

DATA EXTRACTION AND SYNTHESIS

The included articles are concurrently content and thematically analysed using the split approach. Overlapping categories and themes identified are combined using the jigsaw perspective. The themes/categories are compared with the summaries of the included articles in the funnelling process to ensure their accuracy. The domains identified form the framework for the discussion.

RESULTS

12 300 abstracts were reviewed, 946 full-text articles were evaluated and 82 articles were analysed, and the four domains identified were indications, content, design, and strengths and limitations.

CONCLUSIONS

This review reveals that when using a consistent framework, accepted endpoints and outcome measures, longitudinal multisource, multimodal assessment data fashions professional and personal development and enhances identity construction. Future studies into effective assessment tools and support mechanisms are required if portfolio use is to be maximised.

Twelve tips for introducing E-Portfolios in health professions education

Portfolios have been used in health professions for many decades as a means of documenting reflective practice that inform change, supports the understanding of professionals' development needs and changing care options for clients. Electronic versions of one's portfolio of evidence or E-Portfolios became more prevalent in the early 2000s as a repository to store evidence and reflections. However, in recent years E-Portfolios have evolved from a repository to an articulation of authentic learning and development. Introduction of a range of E-Portfolio technology options, hosting systems and increasing professional/ethical standards, has resulted in challenges and opportunities for academics and professionals to meet increasing requirements for teaching and support of students training for health professions. This paper explores twelve tips explained and justified in a Health Science context, designed to support students developing and using E-Portfolios to fulfil professional standards, show evidence of reflective practice and culminate in securing a job in their field.

A systematic scoping review of reflective writing in medical education

BACKGROUND

Reflective writing (RW) allows physicians to step back, review their thoughts, goals and actions and recognise how their perspectives, motives and emotions impact their conduct. RW also helps physicians consolidate their learning and boosts their professional and personal development. In the absence of a consistent approach and amidst growing threats to RW's place in medical training, a review of theories of RW in medical education and a review to map regnant practices, programs and assessment methods are proposed.

METHODS

A Systematic Evidence-Based Approach guided Systematic Scoping Review (SSR in SEBA) was adopted to guide and structure the two concurrent reviews. Independent searches were carried out on publications featured between 1st January 2000 and 30th June 2022 in PubMed, Embase, PsychINFO, CINAHL, ERIC, ASSIA, Scopus, Google Scholar, OpenGrey, GreyLit and ProQuest. The Split Approach saw the included articles analysed separately using thematic and content analysis. Like pieces of a jigsaw puzzle, the Jigsaw Perspective combined the themes and categories identified from both reviews. The Funnelling Process saw the themes/categories created compared with the tabulated summaries. The final domains which emerged structured the discussion that followed.

RESULTS

A total of 33,076 abstracts were reviewed, 1826 full-text articles were appraised and 199 articles were included and analysed. The domains identified were theories and models, current methods, benefits and shortcomings, and recommendations.

CONCLUSIONS

This SSR in SEBA suggests that a structured approach to RW shapes the physician's belief system, guides their practice and nurtures their professional identity formation. In advancing a theoretical concept of RW, this SSR in SEBA proffers new insight into the process of RW, and the need for longitudinal, personalised feedback and support.

Everybody is able to reflect, or aren't they? Evaluating the development of medical professionalism via a longitudinal portfolio mentoring program from a student perspective

Introduction: Reflective competence is fundamental for responsible medical practice and must be systematically incorporated in medical training. To promote this, a longitudinal portfolio-based mentoring program was made mandatory at the Medical Faculty of the University of Tübingen in 2013. This study examines medical students' attitudes toward professional reflection and toward the program in general to draw conclusions about conditions as well as the needs-based design of the program. Method: In winter semester 2017/18, a retrospective questionnaire survey with free text fields was conducted (total sample: N=1.405; students S 1-9; response 37%; S 1-4 "Pre-clinic": n=231; S 5-9 "Clinic": n=241). Opinion trends of semester groups were identified through seven semi-structured interviews with semester speaker and peer tutors. Results: Differences in understanding and attitudes resulted in three positions: 1=approval, 2=ambivalence, 3=rejection. All three groups included individuals from pre-clinical and clinical settings with varying levels of experience. Prior experience and hidden curriculum influenced the position. Opinion trends confirmed the feedback. Conclusion: Although reflection appears in the National Competence-based Learning Objectives Catalogue for Medicine (NKLM), reflective competence is not regarded as equivalent to other study content. Motivation, commitment on the part of the mentors, and a trusting mentor-mentee relationship are effective. The flexibility of the portfolio in terms of content and methodology, as well as the curricular integration of the program are also beneficial.

Medical Student Portfolios: A Systematic Scoping Review

PHENOMENON

Medical Student Portfolios (MSP)s allow medical students to reflect and better appreciate their clinical, research and academic experiences which promotes their individual personal and professional development. However, differences in adoption rate, content design and practice setting create significant variability in their employ. With MSPs increasingly used to evaluate professional competencies and the student's professional identity formation (PIF), this has become an area of concern.

APPROACH

We adopt Krishna's Systematic Evidence-Based Approach to carry out a Systematic Scoping Review (SSR in SEBA) on MSPs. The structured search process of six databases, concurrent use of thematic and content analysis in the Split Approach and comparisons of the themes and categories with the tabulated summaries of included articles in the Jigsaw Perspective and Funnelling Process offers enhanced transparency and reproducibility to this review.

FINDINGS

The research team retrieved 14501 abstracts, reviewed 779 full-text articles and included 96 articles. Similarities between the themes, categories and tabulated summaries allowed the identification of the following funnelled domains: Purpose of MSPs, Content and structure of MSPs, Strengths and limitations of MSPs, Methods to improve MSPs, and Use of E-portfolios.

INSIGHTS

Variability in the employ of MSPs arise as a result of a failure to recognise its different roles and uses. Here we propose additional roles of MSPs, in particular, building on a consistent set of content materials and assessments of milestones called micro-competencies. Whislt generalised micro-competencies assess achievement of general milestones expected of all medical students, personalised micro-competencies record attainment of particular skills, knowledge and attitudes balanced against the medical student's abilities, context and needs. This combination of micro-competencies in a consistent framework promises a holistic, authentic and longitudinal perspective of the medical student's development and maturing PIF.

Reading the Self: Medical Students' Experience of Reflecting on Their Writing Over Time

PURPOSE

To investigate students' experience (over time) with meta-reflection writing exercises, called Signature Reflections. These exercises were used to strengthen reflective capacity, as part of a 4-year reflective writing portfolio curriculum that builds on a recognized strategy for reflection (narrative medicine) and employs longitudinal faculty-mentors.

METHOD

In 2018, the authors conducted 5 focus groups with 18 third-year students from the Columbia University Vagelos College of Physicians and Surgeons class of 2019 to examine students' experience with Signature Reflections. Using an iterative, thematic approach, they developed codes to reflect common patterns in the transcripts, distilled conceptually similar codes, and assembled the code categories into themes.

RESULTS

Three core themes (safe space, narrative experience, mirror of self) and 1 overarching theme (moving through time) were identified. Students frequently experienced relief at having a safe reflective space that promoted grappling with their fears or vulnerabilities and highlighted contextual factors (e.g., trusted faculty-mentors, protected time) that fostered a safe space for reflection and exploration. They often emphasized the value of tangible documentation of their medical school journey (narrative experience) and reported using Signature Reflections to examine their emerging identity (mirror of self). Overlapping with the core themes was a deep appreciation for the temporal perspective facilitated by the Signature Reflections (moving through time).

CONCLUSIONS

A longitudinal narrative medicine-based portfolio curriculum with pauses for meta-reflection allowed students, with faculty support, to observe their trajectory through medical school, explore fears and vulnerabilities, and narrate their own growth. Findings suggest that narrative medicine curricula should be required and sufficiently longitudinal to facilitate opportunities to practice the skill of writing for insight, foster relationships with faculty, and strengthen students' temporal perspectives of their development.

Construct Validity of an Instrument for Assessment of Reflective Writing-Based Portfolios of Medical Students

PURPOSE

Assessment of reflective writing for medical students is challenging, and there is lack of an available instrument with good psychometric properties. The authors developed a new instrument for assessment of reflective writing-based portfolios and examined the construct validity of this instrument.

METHODS

After an extensive literature review and pilot testing of the instrument, two raters assessed the reflective writing-based portfolios from years 2 and 3 medical students (n=135) on three occasions. The instrument consists of three criteria: organization, description of an experience and reflection on the experience. We calculated the reliability of scores using generalizability theory with a fully crossed design and two facets (raters and occasions). In addition, we measured criterion validity by testing correlations with students' scores using other assessment methods.

RESULTS

The dependability (Φ) coefficient of the portfolio scores was 0.75 using two raters on three occasions. Students' portfolio scores represented 46.6% of the total variance across all score comparisons. The variance due to occasions was negligible, while the student-occasion interaction was small. The variance due to student-rater interaction represented 17.7%, and the remaining 27.7% of the variance was due to unexplained sources of error. The decision (D) study suggested that an acceptable dependability (Φ = 0.70 and 0.72) can be achieved by using two raters for one and two occasions, respectively. Finally, we found moderate to large effect-size correlations between students' scores in reflective writing-based portfolios and communication skills (r = 0.47) and PBL tutorials (r = 0.50).

CONCLUSION

We demonstrated the presence of different sources of evidence that support construct validity of the study instrument. Further studies are warranted before utilizing this instrument for summative assessment of students' reflective writing-based portfolios in other medical schools.

Multiple-role mentoring: mentors' conceptualisations, enactments and role conflicts

INTRODUCTION

Outcome-based approaches to education and the inherent emphasis on programmatic assessment in particular, require models of mentoring in which mentors fulfil dual roles: coach and assessor. Fulfilling multiple roles could result in role confusion or even role conflicts, both of which may affect mentoring processes and outcomes. In this study, we explored how mentors conceptualise and enact their role in a multiple-role mentoring system and to what extent they experience role conflicts.

METHODS

We conducted a constructivist grounded theory study at one undergraduate medical school. A purposive sample of 12 physician-mentors active in a programmatic assessment system was interviewed. Data analysis followed stages of open, axial and selective coding through which themes were constructed.

RESULTS

Three predominant mentoring approaches were constructed: (i) empowering (a reflective and holistic approach to student development); (ii) checking (an observant approach to check whether formal requirements are met), and (iii) directing (an authoritative approach to guide students' professional development). Each approach encompassed a corresponding type of mentor-mentee relationship: (i) partnership; (ii) instrumental, and (iii) faculty-centred. Furthermore, mentors' strategies, focus, agency provided to students and perception of the assessment system characterised mentoring approaches and relationships. Role conflicts were mainly experienced by mentors with a directing mentoring approach. They used various coping mechanisms, including deviation from assessment guidelines.

CONCLUSIONS

In multiple-role mentoring in the context of programmatic assessment, mentors adopted certain predominant mentoring approaches, which were characterised by different strategies for mentoring and resulted in different mentor-mentee relationships. Multiple-role mentoring does not necessarily result in role conflict. Mentors who do experience role conflict seem to favour the directing approach, which is most at odds with key principles of competency-based education and programmatic assessment. These findings build upon existing mentoring literature and offer practical suggestions for faculty development regarding approaches to mentoring in programmatic assessment systems.

A Best Evidence in Medical Education systematic review to determine the most effective teaching methods that develop reflection in medical students: BEME Guide No. 51

INTRODUCTION

Reflection is thought to be an essential skill for physicians. Although much has been written about it, there is little concurrence about how to best teach reflection in medical education. The aim of this review was to determine: (i) which educational interventions are being used to develop reflection, (ii) how is reflection being assessed, and (iii) what are the most effective interventions.

METHODS

Inclusion criteria comprised: (i) undergraduate medical students, (ii) a teaching intervention to develop reflection, and (iii) assessment of the intervention. A review protocol was developed and nine databases were searched. Screening, data extraction, and analysis procedures were performed in duplicate. Due to the heterogeneity of studies, a narrative synthesis approach was performed for the study analysis.

RESULTS

Twenty-eight studies met the inclusion criteria. The interventions in these studies had at least of two of the following components related to reflection: (i) introduction, (ii) trigger, (iii) writing, (iv) guidelines, (v) small group discussion, (vi) tutor and (vii) feedback. Three validated rubrics were used to assess reflective writing in these studies.

CONCLUSIONS

The strongest evidence from studies in this review indicates that guidelines for, and feedback on, reflective writing improve student reflection.

Do portfolios have a future?

While portfolios have seen an unprecedented surge in popularity, they have also become the subject of controversy: learners often perceive little gain from writing reflections as part of their portfolios; scholars question the ethics of such obligatory reflection; and students, residents, teachers and scholars alike condemn the bureaucracy surrounding portfolio implementation in competency-based education. It could be argued that mass adoption without careful attention to purpose and format may well jeopardize portfolios' viability in health sciences education. This paper explores this proposition by addressing the following three main questions: (1) Why do portfolios meet with such resistance from students and teachers, while educators love them?; (2) Is it ethical to require students to reflect and then grade their reflections?; (3) Does competency-based education empower or hamper the learner during workplace-based learning? Twenty-five years of portfolio reveal a clear story: without mentoring, portfolios have no future and are nothing short of bureaucratic hurdles in our competency-based education programs. Moreover, comprehensive portfolios, which are integrated into the curriculum and much more diverse in content than reflective portfolios, can serve as meaningful patient charts, providing doctor and patient with useful information to discuss well-being and treatment. In this sense, portfolios are also learner charts that comprehensively document progress in a learning trajectory which is lubricated by meaningful dialogue between learner and mentor in a trusting relationship to foster learning. If we are able to make such comprehensive and meaningful use of portfolios, then, yes, portfolios do have a bright future in medical education.