How do medical schools influence their students' career choices? A realist evaluation

INTRODUCTION

The career choices of medical graduates vary widely between medical schools in the UK and elsewhere and are generally not well matched with societal needs. Research has found that experiences in medical school including formal, informal and hidden curricula are important influences. We conducted a realist evaluation of how and why these various social conditions in medical school influence career thinking.

METHODS

We interviewed junior doctors at the point of applying for speciality training. We selected purposively for a range of career choices. Participants were asked to describe points during their medical training when they had considered career options and how their thinking had been influenced by their context. Interview transcripts were coded for context-mechanism-outcome (CMO) configurations to test initial theories of how career decisions are made.

RESULTS

A total of 26 junior doctors from 12 UK medical schools participated. We found 14 recurring CMO configurations in the data which explained influences on career choice occurring during medical school.

DISCUSSION

Our initial theories about career decision-making were refined as follows: It involves a process of testing for fit of potential careers. This process is asymmetric with multiple experiences needed before deciding a career fits ('easing in') but sometimes only a single negative experience needed for a choice to be ruled out. Developing a preference for a speciality aligns with Person-Environment-Fit decision theories. Ruling out a potential career can however be a less thought-through process than rationality-based decision theories would suggest. Testing for fit is facilitated by longer and more authentic undergraduate placements, allocation of and successful completion of tasks, being treated as part of the team and enthusiastic role models. Informal career guidance is more influential than formal. We suggest some implications for medical school programmes.

How well do UK assistantships equip medical students for graduate practice? Think EPAs

The goal of better medical student preparation for clinical practice drives curricular initiatives worldwide. Learning theory underpins Entrustable Professional Activities (EPAs) as a means of safe transition to independent practice. Regulators mandate senior assistantships to improve practice readiness. It is important to know whether meaningful EPAs occur in assistantships, and with what impact. Final year students at one UK medical school kept learning logs and audio-diaries for six one-week periods during a year-long assistantship. Further data were also obtained through interviewing participants when students and after three months as junior doctors. This was combined with data from new doctors from 17 other UK schools. Realist methods explored what worked for whom and why. 32 medical students and 70 junior doctors participated. All assistantship students reported engaging with EPAs but gaps in the types of EPAs undertaken exist, with level of entrustment and frequency of access depending on the context. Engagement is enhanced by integration into the team and shared understanding of what constitutes legitimate activities. Improving the shared understanding between student and supervisor of what constitutes important assistantship activity may result in an increase in the amount and/or quality of EPAs achieved.

How do medical students' experiences inform their opinions of general practice and its potential as a future career choice?

Recruitment of general practitioners (GPs) in the UK has been suboptimal. There is a wealth of literature exploring recruitment into general practice and opinions of general practice, set in many contexts. This paper aims to synthesise all the elements and extract mechanisms to explain what works, for whom in what circumstances, to create a comprehensive understanding and highlight potential target areas for improvement to improve understanding and fuel interest in general practice careers. A realist review of literature and contemporary data was conducted in line with RAMESES standards. Analysis and synthesis were aided by NVivo. An initial program theory was developed following discussion with expert informers. Sections of text from included literature relating to program theory were extracted and synthesised into a final program theory using a realist logic of analysis in which recurring Context-Mechanism-Outcome configurations (CMOCs) were identified. Searches identified 27 relevant documents. Twenty recurring CMOCs were extracted, explaining how student perceptions of General Practice were influenced by placement experiences, comments from hospital doctors, medical school faculty, peers, GPs, student perceptions of lifestyle and the media. The GP placement environment plays an important role in the formation of opinions about the speciality. The impact of social influence varies depending on the student's desire to belong to a reference group and their level of self-monitoring behaviour. The 'GP lifestyle' was judged to be compatible with many students' needs. This realist review provides recommendations on how curricula can adapt to provide an accurate insight into general practice.

Can learning from workplace feedback be enhanced by reflective writing? A realist evaluation in UK undergraduate medical education

Introduction: Doctors and medical students in the UK are currently required to provide evidence of learning by reflective writing on (among other things) feedback from colleagues. Although the theoretical value of reflecting-on-action is clear, research is still needed to know how to realise the potential of written reflection in medical education. This study arose out of efforts to improve medical student engagement with a reflective writing exercise. We used realist methodology to explain the disinclination of the majority to do written reflection on workplace feedback, and the benefits to the minority.Method: Realist evaluation is a suitable approach to researching complex interventions which have worked for some and not for others. Focus groups were held over a three-year period with year 3 and 4 students. Focus group transcripts were coded for context-mechanism-outcome configurations (the realist approach to analysing data) explaining students' choice not to write a reflection, to write a 'tick-box' reflection or to write for learning. A sub-set of eight students' reflections were also analysed to ascertain evidence of learning through reflection.Results and discussion: 27 students participated in 4 focus groups. Three summary theories emerged showing the importance of context. Firstly, written reflection is effortful and benefits those who invest in it for intrinsic reasons in situations when they need to think more deeply about a learning event. Secondly, following a reflective feedback discussion writing a reflection may add little because the learning has already taken place. Thirdly, external motivation tends to result in writing a 'tick-box' reflection.

Understanding and developing procedures for video-based assessment in medical education

INTRODUCTION

Novel uses of video aim to enhance assessment in health-professionals education. Whilst these uses presume equivalence between video and live scoring, some research suggests that poorly understood variations could challenge validity. We aimed to understand examiners' and students' interaction with video whilst developing procedures to promote its optimal use.

METHODS

Using design-based research we developed theory and procedures for video use in assessment, iteratively adapting conditions across simulated OSCE stations. We explored examiners' and students' perceptions using think-aloud, interviews and focus group. Data were analysed using constructivist grounded-theory methods.

RESULTS

Video-based assessment produced detachment and reduced volitional control for examiners. Examiners ability to make valid video-based judgements was mediated by the interaction of station content and specifically selected filming parameters. Examiners displayed several judgemental tendencies which helped them manage videos' limitations but could also bias judgements in some circumstances. Students rarely found carefully-placed cameras intrusive and considered filming acceptable if adequately justified.

DISCUSSION

Successful use of video-based assessment relies on balancing the need to ensure station-specific information adequacy; avoiding disruptive intrusion; and the degree of justification provided by video's educational purpose. Video has the potential to enhance assessment validity and students' learning when an appropriate balance is achieved.

Clinical assessors' working conceptualisations of undergraduate consultation skills: a framework analysis of how assessors make expert judgements in practice

Undergraduate clinical assessors make expert, multifaceted judgements of consultation skills in concert with medical school OSCE grading rubrics. Assessors are not cognitive machines: their judgements are made in the light of prior experience and social interactions with students. It is important to understand assessors' working conceptualisations of consultation skills and whether they could be used to develop assessment tools for undergraduate assessment. To identify any working conceptualisations that assessors use while assessing undergraduate medical students' consultation skills and develop assessment tools based on assessors' working conceptualisations and natural language for undergraduate consultation skills. In semi-structured interviews, 12 experienced assessors from a UK medical school populated a blank assessment scale with personally meaningful descriptors while describing how they made judgements of students' consultation skills (at exit standard). A two-step iterative thematic framework analysis was performed drawing on constructionism and interactionism. Five domains were found within working conceptualisations of consultation skills: Application of knowledge; Manner with patients; Getting it done; Safety; and Overall impression. Three mechanisms of judgement about student behaviour were identified: observations, inferences and feelings. Assessment tools drawing on participants' conceptualisations and natural language were generated, including 'grade descriptors' for common conceptualisations in each domain by mechanism of judgement and matched to grading rubrics of Fail, Borderline, Pass, Very good. Utilising working conceptualisations to develop assessment tools is feasible and potentially useful. Work is needed to test impact on assessment quality.

'What do we do, doctor?' Transitions of identity and responsibility: a narrative analysis

Transitioning from student to doctor is notoriously challenging. Newly qualified doctors feel required to make decisions before owning their new identity. It is essential to understand how responsibility relates to identity formation to improve transitions for doctors and patients. This multiphase ethnographic study explores realities of transition through anticipatory, lived and reflective stages. We utilised Labov's narrative framework (Labov in J Narrat Life Hist 7(1-4):395-415, 1997) to conduct in-depth analysis of complex relationships between changes in responsibility and development of professional identity. Our objective was to understand how these concepts interact. Newly qualified doctors acclimatise to their role requirements through participatory experience, perceived as a series of challenges, told as stories of adventure or quest. Rules of interaction within clinical teams were complex, context dependent and rarely explicit. Students, newly qualified and supervising doctors felt tensions around whether responsibility should be grasped or conferred. Perceived clinical necessity was a common determinant of responsibility rather than planned learning. Identity formation was chronologically mismatched to accepting responsibility. We provide a rich illumination of the complex relationship between responsibility and identity pre, during, and post-transition to qualified doctor: the two are inherently intertwined, each generating the other through successful actions in practice. This suggests successful transition requires a supported period of identity reconciliation during which responsibility may feel burdensome. During this, there is a fine line between too much and too little responsibility: seemingly innocuous assumptions can have a significant impact. More effort is needed to facilitate behaviours that delegate authority to the transitioning learner whilst maintaining true oversight.

Comparing the influence of 'describing findings to the examiner' or 'examining as in usual practice' on the students' performance and assessors' judgements during physical examination skills assessment

This article was migrated. The article was marked as recommended.

BACKGROUND

Within assessment of physical examination skills, two approaches are common: "Describing Findings" (students comment throughout); and examining as "Usual Practice" (students only report findings at the end). Despite numerous potential influences on both students' performances and assessors' judgements, no prior studies have investigated the influence of either approach on assessments.

METHODS

Two group, randomised, crossover design. Within a 2-station simulated physical examination OSCE, we manipulated whether students "described findings" or examined as "usual practice", collecting 1/. performance scores; 2/. Students'/examiners' cognitive load ratings; ratings of the 3/. fluency and 4/. completeness of students' presentations and 5/. Students' task-finishing, comparing all 5 end-points across conditions.

RESULTS

Neither students' performance scores nor examiners' cognitive load were influenced by experimental condition. Students reported higher cognitive load (7/9) when "describing findings" than "usual practice" (6/9, p=0.002), and were less likely to finish (4 vs 12, p=0.007). Presentation completeness was higher for "describing findings" (mean=2.40, (95CIs=2.05-2.74)) than "usual practice" (mean=1.92 (1.65-2.18),p=0.016), whilst fluency ratings showed a similar trend.

CONCLUSIONS

The decision to "Describe Findings" or examine as "Usual Practice" does not appear neutral, potentially influencing students' efficiency, recall and (by inference) learning. Institutions should explicitly select one option based on assessment goals.

Qualitative research using realist evaluation to explain preparedness for doctors' memorable 'firsts'

CONTEXT

Doctors must be competent from their first day of practice if patients are to be safe. Medical students and new doctors are acutely aware of this, but describe being variably prepared.

OBJECTIVES

This study aimed to identify causal chains of the contextual factors and mechanisms that lead to a trainee being capable (or not) of completing tasks for the first time.

METHODS

We studied three stages of transition: anticipation; lived experience, and post hoc reflection. In the anticipation stage, medical students kept logbooks and audio diaries and were interviewed. Consenting participants were followed into their first jobs as doctors, during which they made audio diaries to capture the lived experiences of transition. Reflection was captured using interviews and focus groups with other postgraduate trainee doctors. All materials were transcribed and references to first experiences ('firsts') were analysed through the lens of realist evaluation.

RESULTS

A total of 32 medical students participated. Eleven participants were followed through the transition to the role of doctor. In addition, 70 postgraduate trainee doctors from three local hospitals who were graduates of 17 UK medical schools participated in 10 focus groups. We identified three categories of firsts (outcomes): firsts that were anticipated and deliberately prepared for in medical school; firsts for which total prior preparedness is not possible as a result of the step change in responsibility between the student and doctor identities, and firsts that represented experiences of failure. Helpful interventions in preparation (context) were opportunities for rehearsal and being given responsibility as a student in the clinical team. Building self-efficacy for tasks was an important mechanism. During transition, the key contextual factor was the provision of appropriate support from colleagues.

CONCLUSIONS

Transition is a step change in responsibility for which total preparedness is not achievable. This transition is experienced as a rite of passage when the newly qualified doctor first makes decisions alone. This study extends the existing literature by explaining the mechanisms involved in preparedness for firsts.

What's in a name? A discussion of what medical students and junior doctors call their senior colleagues, and why it matters

This article was migrated. The article was marked as recommended. How should a medical student address their clinical tutor? Sociolinguistic ideas such as politeness theory tell us that the choice of formal or informal terms of address is determined by the positions of those communicating on two axes; relative status and degree of intimacy. This positioning is influenced by the interaction of personal characteristics of the individuals involved, but there are cross-cultural variations to these rules which are also changing as the world changes. The purpose of the communication will also influence terms of address. There is evidence that reducing social distance within teams improves team-working and that the perception of hierarchy prevents medical students asking for help. Such evidence forces us to take an honest look at how we train our junior colleagues to address us. Students may discover that the etiquette of the medical school classroom differs from that of the clinical placement and find themselves uncertain about how to address their colleagues appropriately. We suggest that it may be helpful in such a quandary to 'mind the gap' rather than ignoring it or trying to close it by imposing a blanket rule on it. We conclude by calling for sociological study with healthcare professionals and their students to discover whether formal or informal forms of address help or hinder aspects of learning and clinical teamwork.

Utility of an app-based system to improve feedback following workplace-based assessment

OBJECTIVES

To determine whether an app-based software system to support production and storage of assessment feedback summaries makes workplace-based assessment easier for clinical tutors and enhances the educational impact on medical students.

METHODS

We monitored our workplace assessor app's usage by Year 3 to 5 medical students in 2014-15 and conducted focus groups with Year 4 medical students and interviews with clinical tutors who had used the apps. Analysis was by constant comparison using a framework based on elements of van der Vleuten's utility index.

RESULTS

The app may enhance the content of feedback for students. Using a screen may be distracting if the app is used during feedback discussions.    Educational impact was reduced by students' perceptions that an easy-to-produce feedback summary is less valuable than one requiring more tutor time and effort. Tutors' typing, dictation skills and their familiarity with mobile devices varied. This influenced their willingness to use the assessment and feedback mobile app rather than the equivalent web app. Electronic feedback summaries had more real and perceived uses than anticipated both for tutors and students including perceptions that they were for the school rather than the student.

CONCLUSIONS

Electronic workplace-based assessment systems can be acceptable to tutors and can make giving detailed written feedback more practical but can interrupt the social interaction required for the feedback conversation. Tutor training and flexible systems will be required to minimise unwanted consequences. The educational impact on both tutors and students of providing pre-formulated advice within the app is worth further study.

Does formal workplace based assessment add value to informal feedback?

This article was migrated. The article was marked as recommended. Feedback is a key component of learning but effective feedback is a complex process with many aspects. One aspect may be a written summary which is passed to the learner but this may not be valued by learners. We examined the role of written feedback in the feedback process to determine whether it does more than provide a simple summary of the interaction. We conducted a secondary analysis of data gathered for a study of formative workplace based assessment. Interview data from 24 interviews with students and written summaries of workplace based assessments for 23 of them were reanalysed by two researchers who were already immersed in the data and examined all references to verbal, informal feedback and written, formal feedback or the assessment tool used. We found that students valued the verbal feedback discussion highly and that they often considered the written summaries superfluous. We also found that the act of preparing written feedback augmented the feedback discussion and tutors had adopted the language of the formal instrument in the verbal feedback and free text written feedback. What this study adds to existing research is evidence that there may be a secondary faculty development effect of requiring the preparation of written feedback which has served to enhance the educational content of feedback. Although this is not proof of causality (the requirement to provide written feedback alone producing the positive effects), we consider that the likelihood is sufficiently strong to continue the practice.

Medical students' perceptions of bedside teaching

BACKGROUND

Bedside teaching is recognised as a valuable tool in medical education by both students and faculty members. Bedside teaching is frequently delivered by consultants; however, junior doctors are increasingly engaging in this form of clinical teaching, and their value in this respect is becoming more widely recognised. The aim of this study was to supplement work completed by previous authors who have begun to explore students' satisfaction with bedside teaching, and their perceptions of the relationship with the clinical teachers. Specifically, we aimed to identify how students perceive bedside teaching delivered by junior doctors compared with consultants. We aimed to identify how students perceived bedside teaching delivered by junior doctors compared with consultants METHODS: A questionnaire was distributed to all third-year medical students at Keele University via e-mail. Responses were submitted anonymously.

RESULTS

Forty-six students responded (37.4%), 73.3 per cent of whom said that they felt more comfortable having bedside teaching delivered by junior doctors than by consultants. Consultants were perceived as more challenging by 60 per cent of respondents. Students appeared to value feedback on their performance, trust the validity of taught information, and to value the overall educational experience equally, regardless of the clinical grade of the teacher.

DISCUSSION

Student preference does not equate to the value that they place on their bedside teaching. Junior doctors are perceived as being more in touch with students and the curriculum, whereas consultants are perceived as having higher expectations and as being both stricter and more knowledgeable. The clinical teacher's approachable manner and enthusiasm for teaching are more important than clinical grade, as is the ability to deliver well-structured constructive feedback.

Guidelines: the do's, don'ts and don't knows of feedback for clinical education

INTRODUCTION

The guidelines offered in this paper aim to amalgamate the literature on formative feedback into practical Do's, Don'ts and Don't Knows for individual clinical supervisors and for the institutions that support clinical learning.

METHODS

The authors built consensus by an iterative process. Do's and Don'ts were proposed based on authors' individual teaching experience and awareness of the literature, and the amalgamated set of guidelines were then refined by all authors and the evidence was summarized for each guideline. Don't Knows were identified as being important questions to this international group of educators which if answered would change practice. The criteria for inclusion of evidence for these guidelines were not those of a systematic review, so indicators of strength of these recommendations were developed which combine the evidence with the authors' consensus.

RESULTS

A set of 32 Do and Don't guidelines with the important Don't Knows was compiled along with a summary of the evidence for each. These are divided into guidelines for the individual clinical supervisor giving feedback to their trainee (recommendations about both the process and the content of feedback) and guidelines for the learning culture (what elements of learning culture support the exchange of meaningful feedback, and what elements constrain it?)

CONCLUSION

Feedback is not easy to get right, but it is essential to learning in medicine, and there is a wealth of evidence supporting the Do's and warning against the Don'ts. Further research into the critical Don't Knows of feedback is required. A new definition is offered: Helpful feedback is a supportive conversation that clarifies the trainee's awareness of their developing competencies, enhances their self-efficacy for making progress, challenges them to set objectives for improvement, and facilitates their development of strategies to enable that improvement to occur.

Grades in formative workplace-based assessment: a study of what works for whom and why

CONTEXT

Grades are commonly used in formative workplace-based assessment (WBA) in medical education and training, but may draw attention away from feedback about the task. A dilemma arises because the self-regulatory focus of a trainee must include self-awareness relative to agreed standards, which implies grading.

OBJECTIVES

In this study we aimed to understand the meaning which medical students construct from WBA feedback with and without grades, and what influences this.

METHODS

Year 3 students were invited to take part in a randomised crossover study in which each student served as his or her own control. Each student undertook one WBA with and one without grades, and then chose whether or not to be given grades in a third WBA. These preferences were explored in semi-structured interviews. A realist approach to analysis was used to gain understanding of student preferences and the impact of feedback with and without grades.

RESULTS

Of 83 students who were given feedback with and without grades, 65 (78%) then chose to have feedback with grades and 18 (22%) without grades in their third WBA. A total of 24 students were interviewed. Students described how grades locate their performance and calibrate their self-assessment. For some, low grades focused attention and effort. Satisfactory and high grades enhanced self-efficacy.

CONCLUSIONS

Grades are concrete, powerful and blunt, can be harmful and need to be explained to help students create helpful meaning from them. Low grades risk reducing self-efficacy in some and may encourage others to focus on proving their ability rather than on improvement. A metaphor of the semi-permeable membrane is introduced to elucidate how students reduced potential negative effects and enhanced the positive effects of feedback with grades by selective filtering and pumping. This study illuminates the complexity of the processing of feedback by its recipients, and informs the use of grading in the provision of more effective, tailored feedback.

Learning the difficult tasks of caring for dying children

None

Development and face validation of strategies for improving consultation skills

While formative workplace based assessment can improve learners' skills, it often does not because the procedures used do not facilitate feedback which is sufficiently specific to scaffold improvement. Provision of pre-formulated strategies to address predicted learning needs has potential to improve the quality and automate the provision of written feedback. To systematically develop, validate and maximise the utility of a comprehensive list of strategies for improvement of consultation skills through a process involving both medical students and their clinical primary and secondary care tutors. Modified Delphi study with tutors, modified nominal group study with students with moderation of outputs by consensus round table discussion by the authors. 35 hospital and 21 GP tutors participated in the Delphi study and contributed 153 new or modified strategies. After review of these and the 205 original strategies, 265 strategies entered the nominal group study to which 46 year four and five students contributed, resulting in the final list of 249 validated strategies. We have developed a valid and comprehensive set of strategies which are considered useful by medical students. This list can be immediately applied by any school which uses the Calgary Cambridge Framework to inform the content of formative feedback on consultation skills. We consider that the list could also be mapped to alternative skills frameworks and so be utilised by schools which do not use the Calgary Cambridge Framework.

Minding the gap between communication skills simulation and authentic experience

CONTEXT

Concurrent exposure to simulated and authentic experiences during undergraduate medical education is increasing. The impact of gaps or differences between contemporaneous experiences has not been adequately considered. We address two questions. How do new undergraduate medical students understand contemporaneous interactions with simulated and authentic patients? How and why do student perceptions of differences between simulated and authentic patient interactions shape their learning?

METHODS

We conducted an interpretative thematic secondary analysis of research data comprising individual interviews (n = 23), focus groups (three groups, n = 16), and discussion groups (four groups, n = 26) with participants drawn from two different year cohorts of Year 1 medical students. These methods generated data from 48 different participants, of whom 17 provided longitudinal data. In addition, data from routinely collected written evaluations of three whole Year 1 cohorts (response rates ≥ 88%, n = 378) were incorporated into our secondary analysis dataset. The primary studies and our secondary analysis were conducted in a single UK medical school with an integrated curriculum.

RESULTS

Our analysis identified that students generate knowledge and meaning from their simulated and authentic experiences relative to each other and that the resultant learning differs in quality according to meaning created by comparing and contrasting contemporaneous experiences. Three themes were identified that clarify how and why the contrasting of differences is an important process for learning outcomes. These are preparedness, responsibility for safety, and perceptions of a gap between theory and practice.

CONCLUSIONS

We propose a conceptual framework generated by reframing common metaphors that refer to the concept of the gap to develop educational strategies that might maximise useful learning from perceived differences. Educators need to 'mind' gaps in collaboration with students if synergistic learning is to be constructed from contemporaneous exposure to simulated and authentic patient interactions. The strategies need to be tested in practice by teachers and learners for utility. Further research is needed to understand gaps in other contexts.

Some like it hot: medical student views on choosing the emotional level of a simulation

OBJECTIVES

This study aimed to determine the impact of giving junior medical students control over the level of emotion expressed by a simulated patient (SP) in a teaching session designed to prepare students to handle emotions when interviewing real patients on placements.

METHODS

  Year 1 medical students at Keele University School of Medicine were allowed to set the degree of emotion to be displayed by the SP in their first 'emotional interview'. This innovation was evaluated by mixed methods in two consecutive academic years as part of an action research project, along with other developments in a new communications skills curriculum. Questionnaires were completed after the first and second iterations by students, tutors and SPs. Sixteen students also participated in evaluative focus group discussions at the end of Year 1.

RESULTS

Most students found the 'emotion-setting switch' helpful, both when interviewing the SP and when observing. Student-interviewers were helped by the perception that they had control over the difficulty of the task. Student-observers found it helpful to see the different levels of emotion and to think about how they might empathise with patients. By contrast, some students found the 'control switch' unnecessary or even unhelpful. These students felt that challenge was good for them and preferred not to be given the option of reducing it.

DISCUSSION

The emotional level control was a useful innovation for most students and may potentially be used in any first encounter with challenging simulation. We suggest that it addresses innate needs for competence and autonomy. The insights gained enable us to suggest ways of building the element of choice into such sessions. The disadvantages of choice highlighted by some students should be surmountable by tutor 'scaffolding' of the learning for both student-interviewers and student-observers.