1
|
Galema G, Schönrock-Adema J, Jaarsma DADC, Wietasch GJKG. Patterns of Medical Residents' Preferences for Organizational Socialization Strategies to Facilitate Their Transitions: A Q-study. PERSPECTIVES ON MEDICAL EDUCATION 2024; 13:169-181. [PMID: 38496363 PMCID: PMC10941690 DOI: 10.5334/pme.1189] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/29/2023] [Accepted: 02/19/2024] [Indexed: 03/19/2024]
Abstract
Introduction To facilitate various transitions of medical residents, healthcare team members and departments may employ various organizational socialization strategies, including formal and informal onboarding methods. However, residents' preferences for these organizational socialization strategies to ease their transition can vary. This study identifies patterns (viewpoints) in these preferences. Methods Using Q-methodology, we asked a purposeful sample of early-career residents to rank a set of statements into a quasi-normal distributed grid. Statements were based on previous qualitative interviews and organizational socialization theory. Participants responded to the question, 'What are your preferences regarding strategies other health care professionals, departments, or hospitals should use to optimize your next transition?' Participants then explained their sorting choices in a post-sort questionnaire. We identified different viewpoints based on by-person (inverted) factor analysis and Varimax rotation. We interpreted the viewpoints using distinguishing and consensus statements, enriched by residents' comments. Results Fifty-one residents ranked 42 statements, among whom 36 residents displayed four distinct viewpoints: Dependent residents (n = 10) favored a task-oriented approach, clear guidance, and formal colleague relationships; Social Capitalizing residents (n = 9) preferred structure in the onboarding period and informal workplace social interactions; Autonomous residents (n = 12) prioritized a loosely structured onboarding period, independence, responsibility, and informal social interactions; and Development-oriented residents (n = 5) desired a balanced onboarding period that allowed independence, exploration, and development. Discussion This identification of four viewpoints highlights the inadequacy of one-size-fits-all approaches to resident transition. Healthcare professionals and departments should tailor their socialization strategies to residents' preferences for support, structure, and formal/informal social interaction.
Collapse
Affiliation(s)
- Gerbrich Galema
- University of Groningen and University Medical Center Groningen (UMCG), and member of the Lifelong Learning, Education and Assessment Research Network (LEARN), She is also a resident in anesthesiology at the department of anesthesiology, NL
| | - Johanna Schönrock-Adema
- Wenckebach Institute for Education and Training, University of Groningen and University Medical Center Groningen, and at the Prins Claus Conservatoire, Hanze University of Applied Sciences, Groningen, Furthermore, she is a LEARN member, NL
| | | | - Götz J. K. G. Wietasch
- University of Groningen and University Medical Center Groningen, Department of Anesthesiology, and a LEARN member, NL
| |
Collapse
|
2
|
Mughal Z, Patel S, Gupta KK, Metcalfe C, Beech T, Jennings C. Evaluating the perceptions of workplace-based assessments in surgical training: a systematic review. Ann R Coll Surg Engl 2023; 105:507-512. [PMID: 36374289 PMCID: PMC10313445 DOI: 10.1308/rcsann.2022.0113] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 08/01/2022] [Indexed: 11/24/2023] Open
Abstract
INTRODUCTION Workplace-based assessments (WBAs) are intended to maximise learning opportunities in surgical training. There is speculation as to whether mandatory assessments in this form contribute to a tick-box culture. The objective of this review was to investigate surgical trainees' attitudes towards WBAs. METHODS This systematic review of qualitative studies was performed in accordance with the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) statement. The literature was searched on the Medline®, Embase™, PubMed and Web of Science™ databases on 22 March 2022. RESULTS Sixteen studies were included in the review, mostly carried out on users of the Intercollegiate Surgical Curriculum Programme portfolio in the UK. Trainees felt that WBAs were educationally useful, providing opportunity for feedback, but this was overshadowed by a pressure to reach a set annual quota for WBAs and achieve high scores. Other themes included inaccurate recording of WBAs, the role of WBAs as formative or summative assessments, engagement and accessibility of trainers, and lack of time to complete WBAs. CONCLUSIONS Negative perceptions about WBAs were widespread among surgical trainees despite a recognition of their capacity to facilitate learning. This review supports the recent removal of the annual quota for WBAs in UK surgical training programmes.
Collapse
Affiliation(s)
- Z Mughal
- University Hospitals Birmingham NHS Foundation Trust, UK
| | - S Patel
- Shrewsbury and Telford Hospital NHS Foundation Trust, UK
| | - K Kumar Gupta
- University Hospitals Birmingham NHS Foundation Trust, UK
| | - C Metcalfe
- University Hospitals Birmingham NHS Foundation Trust, UK
| | - T Beech
- University Hospitals Birmingham NHS Foundation Trust, UK
| | - C Jennings
- University Hospitals Birmingham NHS Foundation Trust, UK
| |
Collapse
|
3
|
Leep Hunderfund AN, Santilli AR, Rubin DI, Laughlin RS, Sorenson EJ, Park YS. Assessing electrodiagnostic skills among residents and fellows: Relationships between workplace-based assessments using the Electromyography Direct Observation Tool and other measures of trainee performance. Muscle Nerve 2022; 66:671-678. [PMID: 35470901 DOI: 10.1002/mus.27566] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2021] [Revised: 04/21/2022] [Accepted: 04/23/2022] [Indexed: 12/14/2022]
Abstract
INTRODUCTION/AIMS Graduate medical education programs must ensure residents and fellows acquire skills needed for independent practice. Workplace-based observational assessments are informative but can be time- and resource-intensive. In this study we sought to gather "relations-to-other-variables" validity evidence for scores generated by the Electromyography Direct Observation Tool (EMG-DOT) to inform its use as a measure of electrodiagnostic skill acquisition. METHODS Scores on multiple assessments were compiled by trainees during Clinical Neurophysiology and Electromyography rotations at a large US academic medical center. Relationships between workplace-based EMG-DOT scores (n = 298) and scores on a prerequisite simulated patient exercise, patient experience surveys (n = 199), end-of-rotation evaluations (n = 301), and an American Association of Neuromuscular & Electrodiagnostic Medicine (AANEM) self-assessment examination were assessed using Pearson correlations. RESULTS Among 23 trainees, EMG-DOT scores assigned by physician raters correlated positively with end-of-rotation evaluations (r = 0.63, P = .001), but EMG-DOT scores assigned by technician raters did not (r = 0.10, P = .663). When physician and technician ratings were combined, higher EMG-DOT scores correlated with better patient experience survey scores (r = 0.42, P = .047), but not with simulated patient or AANEM self-assessment examination scores. DISCUSSION End-of-rotation evaluations can provide valid assessments of trainee performance when completed by individuals with ample opportunities to directly observe trainees. Inclusion of observational assessments by technicians and patients provides a more comprehensive view of trainee performance. Workplace- and classroom-based assessments provide complementary information about trainee performance, reflecting underlying differences in types of skills measured.
Collapse
Affiliation(s)
| | - Ashley R Santilli
- Department of Neurology at Mayo Clinic College of Medicine, Mayo Clinic, Rochester, Minnesota
| | - Devon I Rubin
- Department of Neurology at Mayo Clinic College of Medicine, Jacksonville, Florida
| | - Ruple S Laughlin
- Department of Neurology at Mayo Clinic College of Medicine, Mayo Clinic, Rochester, Minnesota
| | - Eric J Sorenson
- Department of Neurology at Mayo Clinic College of Medicine, Mayo Clinic, Rochester, Minnesota
| | - Yoon S Park
- Department of Medical Education, University of Illinois College of Medicine, Chicago, Illinois.,Health Professions Education Research at Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts
| |
Collapse
|
4
|
Desire paths for workplace assessment in postgraduate anaesthesia training: analysing informal processes to inform assessment redesign. Br J Anaesth 2022; 128:997-1005. [DOI: 10.1016/j.bja.2022.03.013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2021] [Revised: 02/24/2022] [Accepted: 03/06/2022] [Indexed: 11/17/2022] Open
|
5
|
Klein BA. Q-Methodology: A Technique to Assess Perceptions in Simulation Research. Simul Healthc 2022; 17:e128-e135. [PMID: 34009909 DOI: 10.1097/sih.0000000000000571] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
SUMMARY STATEMENT Learner perceptions of simulation can be challenging to assess and are usually limited to qualitative strategies. Q-methodology is an exploratory, systematic research technique that combines quantitative and qualitative procedures to measure differences in subjectivity. This article briefly reviews Q-methodology, provides an example of how it could be applied to assess the affective domain of simulation-based learning, and discusses implications for the use of Q-methodology in future simulation scholarship.
Collapse
Affiliation(s)
- Barbie A Klein
- From the Department of Anatomy, University of California, San Francisco, San Francisco, CA. ORCID: https://orcid.org/0000-0003-2845-7136
| |
Collapse
|
6
|
Kusurkar RA, Mak-van der Vossen M, Kors J, Grijpma JW, van der Burgt SME, Koster AS, de la Croix A. 'One size does not fit all': The value of person-centred analysis in health professions education research. PERSPECTIVES ON MEDICAL EDUCATION 2021; 10:245-251. [PMID: 33284407 PMCID: PMC8368141 DOI: 10.1007/s40037-020-00633-w] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/14/2020] [Revised: 10/07/2020] [Accepted: 11/05/2020] [Indexed: 05/10/2023]
Abstract
Health professions education (HPE) research is dominated by variable-centred analysis, which enables the exploration of relationships between different independent and dependent variables in a study. Although the results of such analysis are interesting, an effort to conduct a more person-centred analysis in HPE research can help us in generating a more nuanced interpretation of the data on the variables involved in teaching and learning. The added value of using person-centred analysis, next to variable-centred analysis, lies in what it can bring to the applications of the research findings in educational practice. Research findings of person-centred analysis can facilitate the development of more personalized learning or remediation pathways and customization of teaching and supervision efforts. Making the research findings more recognizable in practice can make it easier for teachers and supervisors to understand and deal with students. The aim of this article is to compare and contrast different methods that can be used for person-centred analysis and show the incremental value of such analysis in HPE research. We describe three methods for conducting person-centred analysis: cluster, latent class and Q‑sort analyses, along with their advantages and disadvantage with three concrete examples for each method from HPE research studies.
Collapse
Affiliation(s)
- Rashmi A Kusurkar
- Amsterdam UMC, Faculty of Medicine, Vrije Universiteit Amsterdam, Research in Education, Amsterdam, The Netherlands.
- LEARN! Research Institute for Learning and Education, Faculty of Psychology and Education, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands.
| | - Marianne Mak-van der Vossen
- Amsterdam UMC, Faculty of Medicine, Vrije Universiteit Amsterdam, Research in Education, Amsterdam, The Netherlands
- LEARN! Research Institute for Learning and Education, Faculty of Psychology and Education, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| | - Joyce Kors
- Amsterdam UMC, Faculty of Medicine, Vrije Universiteit Amsterdam, Research in Education, Amsterdam, The Netherlands
- LEARN! Research Institute for Learning and Education, Faculty of Psychology and Education, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| | - Jan-Willem Grijpma
- Amsterdam UMC, Faculty of Medicine, Vrije Universiteit Amsterdam, Research in Education, Amsterdam, The Netherlands
- LEARN! Research Institute for Learning and Education, Faculty of Psychology and Education, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
- LEARN! Academy, Faculty of Behavioural and Movement Sciences, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| | | | - Andries S Koster
- Department of Pharmaceutical Sciences, Utrecht University, Utrecht, The Netherlands
| | - Anne de la Croix
- Amsterdam UMC, Faculty of Medicine, Vrije Universiteit Amsterdam, Research in Education, Amsterdam, The Netherlands
- LEARN! Research Institute for Learning and Education, Faculty of Psychology and Education, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| |
Collapse
|
7
|
Churruca K, Ludlow K, Wu W, Gibbons K, Nguyen HM, Ellis LA, Braithwaite J. A scoping review of Q-methodology in healthcare research. BMC Med Res Methodol 2021; 21:125. [PMID: 34154566 PMCID: PMC8215808 DOI: 10.1186/s12874-021-01309-7] [Citation(s) in RCA: 34] [Impact Index Per Article: 11.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/18/2021] [Accepted: 04/30/2021] [Indexed: 12/26/2022] Open
Abstract
BACKGROUND Q-methodology is an approach to studying complex issues of human 'subjectivity'. Although this approach was developed in the early twentieth century, the value of Q-methodology in healthcare was not recognised until relatively recently. The aim of this review was to scope the empirical healthcare literature to examine the extent to which Q-methodology has been utilised in healthcare over time, including how it has been used and for what purposes. METHODS A search of three electronic databases (Scopus, EBSCO-CINAHL Complete, Medline) was conducted. No date restriction was applied. A title and abstract review, followed by a full-text review, was conducted by a team of five reviewers. Included articles were English-language, peer-reviewed journal articles that used Q-methodology (both Q-sorting and inverted factor analysis) in healthcare settings. The following data items were extracted into a purpose-designed Excel spreadsheet: study details (e.g., setting, country, year), reasons for using Q-methodology, healthcare topic area, participants (type and number), materials (e.g., ranking anchors and Q-set), methods (e.g., development of the Q-set, analysis), study results, and study implications. Data synthesis was descriptive in nature and involved frequency counting, open coding and the organisation by data items. RESULTS Of the 2,302 articles identified by the search, 289 studies were included in this review. We found evidence of increased use of Q-methodology in healthcare, particularly over the last 5 years. However, this research remains diffuse, spread across a large number of journals and topic areas. In a number of studies, we identified limitations in the reporting of methods, such as insufficient information on how authors derived their Q-set, what types of analyses they performed, and the amount of variance explained. CONCLUSIONS Although Q-methodology is increasingly being adopted in healthcare research, it still appears to be relatively novel. This review highlight commonalities in how the method has been used, areas of application, and the potential value of the approach. To facilitate reporting of Q-methodological studies, we present a checklist of details that should be included for publication.
Collapse
Affiliation(s)
- Kate Churruca
- Centre for Healthcare Resilience and Implementation Science, Australian Institute of Health Innovation, Macquarie University, Level 6, 75 Talavera Rd, Sydney, NSW, 2109, Australia.
| | - Kristiana Ludlow
- School of Psychology, University of Queensland, Brisbane, QLD, 4072, Australia
- Australian Institute of Health Innovation, Macquarie University, Sydney, NSW, 2109, Australia
| | - Wendy Wu
- Centre for Healthcare Resilience and Implementation Science, Australian Institute of Health Innovation, Macquarie University, Level 6, 75 Talavera Rd, Sydney, NSW, 2109, Australia
| | - Kate Gibbons
- Centre for Healthcare Resilience and Implementation Science, Australian Institute of Health Innovation, Macquarie University, Level 6, 75 Talavera Rd, Sydney, NSW, 2109, Australia
| | - Hoa Mi Nguyen
- Centre for Healthcare Resilience and Implementation Science, Australian Institute of Health Innovation, Macquarie University, Level 6, 75 Talavera Rd, Sydney, NSW, 2109, Australia
| | - Louise A Ellis
- Centre for Healthcare Resilience and Implementation Science, Australian Institute of Health Innovation, Macquarie University, Level 6, 75 Talavera Rd, Sydney, NSW, 2109, Australia
| | - Jeffrey Braithwaite
- Centre for Healthcare Resilience and Implementation Science, Australian Institute of Health Innovation, Macquarie University, Level 6, 75 Talavera Rd, Sydney, NSW, 2109, Australia
| |
Collapse
|
8
|
Grijpma JW, de la Croix A, Kleinveld JH, Meeter M, Kusurkar RA. Appreciating small-group active learning: What do medical students want, and why? A Q-methodology study. MEDICAL TEACHER 2021; 43:411-420. [PMID: 33327835 DOI: 10.1080/0142159x.2020.1854705] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
INTRODUCTION For Small-Group Active Learning (SMAL) to be effective, students need to engage meaningfully in learning activities to construct their knowledge. Teachers can have difficulty engaging their students in this process. To improve engagement, we aimed to identify the diversity in medical students' appreciation of SMAL, using the concepts of epistemic beliefs and approaches to learning. METHOD Q-methodology is a mixed-method research design used for the systematic study of subjectivity. We developed a set of 54 statements on active learning methods. In individual interviews, first-year medical students rank ordered their agreement with these statements and explained their reasons. Data were analyzed using a by-person factor analysis to group participants with shared viewpoints. RESULTS A four-factor solution (i.e. profiles) fit the data collected from 52 students best and explained 52% of the variance. Each profile describes a shared viewpoint on SMAL. We characterized the profiles as 'understanding-oriented', 'assessment-oriented', 'group-oriented', and 'practice-oriented'. DISCUSSION The four profiles describe how and why students differ in their appreciation of SMAL. Teachers can use the profiles to make better-informed decisions when designing and teaching their SMAL classes, by relating to students' epistemic beliefs, and approaches to learning. This may improve student motivation and engagement for SMAL.
Collapse
Affiliation(s)
- J W Grijpma
- Amsterdam UMC, Research in Education, Faculty of Medicine, Vrije Universiteit, Amsterdam, the Netherlands
- Faculty of Behavioural and Movement Sciences, Vrije Universiteit Amsterdam, LEARN! Academy, Amsterdam, Netherlands
- Vrije Universiteit Amsterdam, LEARN! Research Institute, Amsterdam, Netherlands
| | - A de la Croix
- Amsterdam UMC, Research in Education, Faculty of Medicine, Vrije Universiteit, Amsterdam, the Netherlands
- Faculty of Behavioural and Movement Sciences, Vrije Universiteit Amsterdam, LEARN! Academy, Amsterdam, Netherlands
- Vrije Universiteit Amsterdam, LEARN! Research Institute, Amsterdam, Netherlands
| | - J H Kleinveld
- Department of Education Strategy and Policy, Faculty of Medicine, Vrije Universiteit Amsterdam, Amsterdam, Netherlands
- Department of Clinical Genetics, Amsterdam University Medical Center, Amsterdam, Netherlands
| | - M Meeter
- Vrije Universiteit Amsterdam, LEARN! Research Institute, Amsterdam, Netherlands
| | - R A Kusurkar
- Amsterdam UMC, Research in Education, Faculty of Medicine, Vrije Universiteit, Amsterdam, the Netherlands
- Vrije Universiteit Amsterdam, LEARN! Research Institute, Amsterdam, Netherlands
| |
Collapse
|
9
|
Prentice S, Benson J, Kirkpatrick E, Schuwirth L. Workplace-based assessments in postgraduate medical education: A hermeneutic review. MEDICAL EDUCATION 2020; 54:981-992. [PMID: 32403200 DOI: 10.1111/medu.14221] [Citation(s) in RCA: 26] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/10/2019] [Revised: 03/30/2020] [Accepted: 05/06/2020] [Indexed: 06/11/2023]
Abstract
OBJECTIVES Since their introduction, workplace-based assessments (WBAs) have proliferated throughout postgraduate medical education. Previous reviews have identified mixed findings regarding WBAs' effectiveness, but have not considered the importance of user-tool-context interactions. The present review was conducted to address this gap by generating a thematic overview of factors important to the acceptability, effectiveness and utility of WBAs in postgraduate medical education. METHOD This review utilised a hermeneutic cycle for analysis of the literature. Four databases were searched to identify articles pertaining to WBAs in postgraduate medical education from the United Kingdom, Canada, Australia, New Zealand, the Netherlands and Scandinavian countries. Over the course of three rounds, 30 published articles were thematically analysed in an iterative fashion to deeply engage with the literature in order to answer three scoping questions concerning acceptability, effectiveness and assessment training. As each round was coded, themes were refined and questions added until saturation was reached. RESULTS Stakeholders value WBAs for permitting assessment of trainees' performance in an authentic context. Negative perceptions of WBAs stem from misuse due to low assessment literacy, disagreement with definitions and frameworks, and inadequate summative use of WBAs. Effectiveness is influenced by user (eg, engagement and assessment literacy) and tool attributes (eg, definitions and scales), but most fundamentally by user-tool-context interactions, particularly trainee-assessor relationships. Assessors' assessment literacy must be combined with cultural and administrative factors in organisations and the broader medical discipline. CONCLUSIONS The pivotal determinants of WBAs' effectiveness and utility are the user-tool-context interactions. From the identified themes, we present 12 lessons learned regarding users, tools and contexts to maximise WBA utility, including the separation of formative and summative WBA assessors, use of maximally useful scales, and instituting measures to reduce competitive demands.
Collapse
Affiliation(s)
- Shaun Prentice
- GPEx Ltd., Adelaide, South Australia, Australia
- School of Psychology, University of Adelaide, Adelaide, South Australia, Australia
| | - Jill Benson
- GPEx Ltd., Adelaide, South Australia, Australia
- Health in Human Diversity Unit, School of Medicine, University of Adelaide, Adelaide, South Australia, Australia
- Prideaux Centre, Flinders University, Adelaide, South Australia, Australia
| | - Emily Kirkpatrick
- GPEx Ltd., Adelaide, South Australia, Australia
- School of Medicine, University of Adelaide, Adelaide, South Australia, Australia
| | - Lambert Schuwirth
- Prideaux Centre, Flinders University, Adelaide, South Australia, Australia
- Maastrich University, Maastricht, the Netherlands
- Uniformed University for the Health Sciences, Bethesda, Maryland, USA
| |
Collapse
|
10
|
Weller JM, Naik VN, San Diego RJ. Systematic review and narrative synthesis of competency-based medical education in anaesthesia. Br J Anaesth 2020; 124:748-760. [DOI: 10.1016/j.bja.2019.10.025] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2019] [Revised: 10/06/2019] [Accepted: 10/29/2019] [Indexed: 11/16/2022] Open
|
11
|
Barrett A, Hanley K. Myths and legends in workplace-based assessment; it’s all about perspective? EDUCATION FOR PRIMARY CARE 2020; 31:74-80. [DOI: 10.1080/14739879.2020.1712664] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
Affiliation(s)
- Aileen Barrett
- GP Training, Irish College of General Practitioners, Dublin, Ireland
| | - Karena Hanley
- GP Training, Irish College of General Practitioners, Dublin, Ireland
| |
Collapse
|
12
|
Kirschbaum M, Barnett T, Cross M. Q sample construction: a novel approach incorporating a Delphi technique to explore opinions about codeine dependence. BMC Med Res Methodol 2019; 19:101. [PMID: 31088351 PMCID: PMC6518693 DOI: 10.1186/s12874-019-0741-9] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2018] [Accepted: 04/26/2019] [Indexed: 02/05/2023] Open
Abstract
BACKGROUND Q methodology is an evidenced approach to researching subjectivity, involving a combination of qualitative and quantitative techniques. The methodology has been used successfully in healthcare research to explore the opinions of patients and healthcare providers about topics such as the illness experience, healthcare services, clinical practice and professional training. Q methodology studies require the generation of a Q sample, a set of opinion statements representing the phenomenon of interest. This paper describes a novel and rigorous approach to develop a Q sample for a study exploring misusers' opinions about over-the-counter (OTC) codeine dependence and critically examines the associated methodological issues. METHODS Development of the Q sample in this study involved three steps; (1) identification of opinion statements via a comprehensive literature search, (2) application of a theoretical framework, the Capability, Opportunity, Motivation - Behaviour (COM-B) model of behaviour, to group and then reduce the number of statements and (3) use of a Delphi technique to achieve expert consensus on the final selection of statements. The Delphi component involved a multidisciplinary panel of 15 addiction experts comprised of doctors, nurses, pharmacists, psychologists and researchers, who were recruited purposively. Experts rated each statement using a 5-point scale of perceived importance. Two Delphi rounds were undertaken and consensus for inclusion of a statement was set at a median score of ≥4 and an interquartile range of ≤1. RESULTS A total of 842 statements representing codeine misusers' opinions about OTC codeine dependence were identified from the literature. Statements were grouped thematically using the COM-B framework and representative statements were selected, reducing the number to 111. After two Delphi rounds, addiction experts achieved consensus on 46 statements which formed the final Q sample. CONCLUSIONS This paper describes a new and systematic approach to Q sample construction and explores associated methodological issues that could be useful for those considering Q methodology and for furthering the rigour of this research technique.
Collapse
Affiliation(s)
- Melissa Kirschbaum
- University of Tasmania Centre for Rural Health, Locked Bag 1372, Launceston, Tasmania 7250 Australia
| | - Tony Barnett
- University of Tasmania Centre for Rural Health, Locked Bag 1372, Launceston, Tasmania 7250 Australia
| | - Merylin Cross
- University of Tasmania Centre for Rural Health, Locked Bag 1372, Launceston, Tasmania 7250 Australia
| |
Collapse
|
13
|
Govaerts MJB, van der Vleuten CPM, Holmboe ES. Managing tensions in assessment: moving beyond either-or thinking. MEDICAL EDUCATION 2019; 53:64-75. [PMID: 30289171 PMCID: PMC6586064 DOI: 10.1111/medu.13656] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/13/2018] [Revised: 04/16/2018] [Accepted: 06/08/2018] [Indexed: 05/09/2023]
Abstract
CONTEXT In health professions education, assessment systems are bound to be rife with tensions as they must fulfil formative and summative assessment purposes, be efficient and effective, and meet the needs of learners and education institutes, as well as those of patients and health care organisations. The way we respond to these tensions determines the fate of assessment practices and reform. In this study, we argue that traditional 'fix-the-problem' approaches (i.e. either-or solutions) are generally inadequate and that we need alternative strategies to help us further understand, accept and actually engage with the multiple recurring tensions in assessment programmes. METHODS Drawing from research in organisation science and health care, we outline how the Polarity Thinking™ model and its 'both-and' approach offer ways to systematically leverage assessment tensions as opportunities to drive improvement, rather than as intractable problems. In reviewing the assessment literature, we highlight and discuss exemplars of specific assessment polarities and tensions in educational settings. Using key concepts and principles of the Polarity Thinking™ model, and two examples of common tensions in assessment design, we describe how the model can be applied in a stepwise approach to the management of key polarities in assessment. DISCUSSION Assessment polarities and tensions are likely to surface with the continued rise of complexity and change in education and health care organisations. With increasing pressures of accountability in times of stretched resources, assessment tensions and dilemmas will become more pronounced. We propose to add to our repertoire of strategies for managing key dilemmas in education and assessment design through the adoption of the polarity framework. Its 'both-and' approach may advance our efforts to transform assessment systems to meet complex 21st century education, health and health care needs.
Collapse
Affiliation(s)
- Marjan J B Govaerts
- Department of Educational Development and ResearchFaculty of Health, Medicine and Life SciencesMaastricht UniversityMaastrichtthe Netherlands
| | - Cees P M van der Vleuten
- Department of Educational Development and ResearchFaculty of Health, Medicine and Life SciencesMaastricht UniversityMaastrichtthe Netherlands
| | - Eric S Holmboe
- Accreditation Council for Graduate Medical EducationChicagoIllinoisUSA
| |
Collapse
|
14
|
Waterval DGJ, Frambach JM, Driessen EW, Muijtjens A, Scherpbier AJJA. Connected, attracted, and concerned: A Q study on medical crossborder curriculum partnerships. MEDICAL TEACHER 2018; 40:1293-1299. [PMID: 29415599 DOI: 10.1080/0142159x.2018.1431618] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
INTRODUCTION A new form of internationalization has been trending upward in the medical education realm: crossborder medical curriculum partnerships established to deliver the same, or adapted, curriculum to groups of geographically separated students. This study aims to investigate crossborder medical curriculum partnerships by exploring the experiences of teachers at the recipient institution who have a key role in delivering the program. METHODS From four pioneering recipient medical schools, 24 teachers participated in a Q-sort study. Each participant rank-ordered 42 statements about teaching in a crossborder medical curriculum on a scale from -5 (indicating strong disagreement) to +5 (indicating strong agreement). The authors conducted a "by-person" factor analysis to uncover distinct patterns in the ranking of statements, using the statistical results and participants' comments about their Q sorts to interpret these patterns and translate them into distinct viewpoints. RESULTS Three viewpoints emerged, reflecting: (1) a feeling of connectedness with the partner institution, trust in the quality of the curriculum, and appreciation of interinstitutional relationships; (2) the partnership's attractiveness because of the career opportunities it offers; and (3) concerns over the quality of graduates because of doubts about the appropriateness of the didactic model and insufficient attention to local healthcare needs, and over the practical feasibility of such partnerships. CONCLUSIONS The three viewpoints identified revealed a pallet of views on how host teachers might experience their work. It shows the heterogeneous features of this group and seems to counterbalance reports that they are feeling "deprived" from their role as teacher. Two viewpoints featured an appreciation of interinstitutional relationships and of the partnership, especially when perceiving a degree of autonomy. Partners can capitalize on all different viewpoints by deploying procedure and policies to raise the quality of education delivery.
Collapse
Affiliation(s)
- Dominique G J Waterval
- a School of Health Professions Education , Maastricht University , Maastricht , The Netherlands
| | - Janneke M Frambach
- a School of Health Professions Education , Maastricht University , Maastricht , The Netherlands
| | - Erik W Driessen
- a School of Health Professions Education , Maastricht University , Maastricht , The Netherlands
| | - Arno Muijtjens
- a School of Health Professions Education , Maastricht University , Maastricht , The Netherlands
| | - Albert J J A Scherpbier
- a School of Health Professions Education , Maastricht University , Maastricht , The Netherlands
| |
Collapse
|
15
|
van Vendeloo SN, Brand PLP, Kollen BJ, Verheyen CCPM. Changes in Perceived Supervision Quality After Introduction of Competency-Based Orthopedic Residency Training: A National 6-Year Follow-Up Study. JOURNAL OF SURGICAL EDUCATION 2018; 75:1624-1629. [PMID: 29706298 DOI: 10.1016/j.jsurg.2018.04.006] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/18/2017] [Revised: 01/28/2018] [Accepted: 04/03/2018] [Indexed: 06/08/2023]
Abstract
OBJECTIVE To evaluate the perceived quality of the learning environment, before and after introduction of competency-based postgraduate orthopedic education. DESIGN From 2009 to 2014, we conducted annual surveys among Dutch orthopedic residents. The validated Dutch Residency Educational Climate Test (D-RECT, 50 items on 11 subscales) was used to assess the quality of the learning environment. Scores range from 1 (poor) to 5 (excellent). SETTING Dynamic cohort follow-up study. PARTICIPANTS All Dutch orthopedic residents were surveyed during annual compulsory courses. RESULTS Over the 6-year period, 641 responses were obtained (response rate 92%). Scores for "supervision" (95% CI for difference 0.06-0.28, p = 0.002) and "coaching and assessment" (95% CI 0.11-0.35, p < 0.001) improved significantly after introduction of competency-based training. There was no significant change in score on the other subscales of the D-RECT. CONCLUSIONS After the introduction of some of the core components of competency-based postgraduate orthopedic education the perceived quality of "supervision" and "coaching and assessment" improved significantly.
Collapse
Affiliation(s)
- Stefan N van Vendeloo
- Department of Orthopaedic Surgery and Traumatology, Isala Hospital, Zwolle, The Netherlands.
| | - Paul L P Brand
- Department of Pediatrics, Princess Amalia Children's Centre, Isala Hospital, Zwolle, The Netherlands; UMCG Postgraduate School of Medicine, University Medical Centre Groningen, University of Groningen, The Netherlands
| | - Boudewijn J Kollen
- Department of General Practice, University Medical Centre Groningen, University of Groningen, The Netherlands
| | - Cees C P M Verheyen
- Department of Orthopaedic Surgery and Traumatology, Isala Hospital, Zwolle, The Netherlands
| |
Collapse
|
16
|
Edirippulige S, Smith AC, Wickramasinghe S, Armfield NR. Examining the Influence of E-Health Education on Professional Practice. J Med Syst 2018; 42:215. [DOI: 10.1007/s10916-018-1084-5] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2018] [Accepted: 09/25/2018] [Indexed: 11/29/2022]
|
17
|
Hoff RG, Frenkel J, Imhof SM, Ten Cate O. Flexibility in Postgraduate Medical Training in the Netherlands. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2018; 93:S32-S36. [PMID: 29485485 DOI: 10.1097/acm.0000000000002078] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
Postgraduate medical training in the Netherlands has become increasingly individualized. In this article, the authors describe current practices for three residency programs at the University Medical Center Utrecht: anesthesiology, pediatrics, and ophthalmology. These programs are diverse yet share characteristics allowing for individualized residency training. New residents enter each program throughout the year, avoiding a large simultaneous influx of inexperienced doctors. The usual duration of each is five years. However, the actual duration of rotations or of the program as a whole can be reduced because of residents' previous medical experience or demonstration of early mastery of relevant competencies. If necessary, the duration of training can also increase.Although working hours are already restricted by the European Working Time Directive, most residents choose to train on a part-time basis. The length of their program then is extended proportionally. The extension period added for those residents training part-time can be used to develop specific competencies, complete an elective rotation or research, or explore a focus area. If the resident meets all training objectives before the extension period is completed, the program director can choose to shorten the program length. Recently, entrustable professional activities have been introduced to strengthen workplace-based assessment. The effects on program duration have yet to be demonstrated.Flexible postgraduate training is feasible. Although improving work-life balance for residents is a necessity, attention must be paid to ensuring that they gain the necessary experience and competencies and maintain continuity of care to ensure that high-quality patient care is provided.
Collapse
Affiliation(s)
- Reinier G Hoff
- R.G. Hoff is professor of education and training in perioperative, intensive, and emergency care and program director, Anesthesiology Residency, Department of Anesthesiology, University Medical Center Utrecht, Utrecht, The Netherlands. J. Frenkel is professor of patient- and family-centered education and program director, Pediatrics Residency, Department of Pediatrics, University Medical Center Utrecht, Utrecht, The Netherlands. S.M. Imhof is professor of ophthalmology and program director, Ophthalmology Residency, Department of Ophthalmology, and chair, Central Residency Committee, University Medical Center Utrecht, Utrecht, The Netherlands. O. ten Cate is professor of medical education, Center for Research and Development of Education, University Medical Center Utrecht, Utrecht, The Netherlands
| | | | | | | |
Collapse
|
18
|
de Jonge LPJWM, Timmerman AA, Govaerts MJB, Muris JWM, Muijtjens AMM, Kramer AWM, van der Vleuten CPM. Stakeholder perspectives on workplace-based performance assessment: towards a better understanding of assessor behaviour. ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2017; 22:1213-1243. [PMID: 28155004 PMCID: PMC5663793 DOI: 10.1007/s10459-017-9760-7] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/07/2016] [Accepted: 01/24/2017] [Indexed: 05/13/2023]
Abstract
Workplace-Based Assessment (WBA) plays a pivotal role in present-day competency-based medical curricula. Validity in WBA mainly depends on how stakeholders (e.g. clinical supervisors and learners) use the assessments-rather than on the intrinsic qualities of instruments and methods. Current research on assessment in clinical contexts seems to imply that variable behaviours during performance assessment of both assessors and learners may well reflect their respective beliefs and perspectives towards WBA. We therefore performed a Q methodological study to explore perspectives underlying stakeholders' behaviours in WBA in a postgraduate medical training program. Five different perspectives on performance assessment were extracted: Agency, Mutuality, Objectivity, Adaptivity and Accountability. These perspectives reflect both differences and similarities in stakeholder perceptions and preferences regarding the utility of WBA. In comparing and contrasting the various perspectives, we identified two key areas of disagreement, specifically 'the locus of regulation of learning' (i.e., self-regulated versus externally regulated learning) and 'the extent to which assessment should be standardised' (i.e., tailored versus standardised assessment). Differing perspectives may variously affect stakeholders' acceptance, use-and, consequently, the effectiveness-of assessment programmes. Continuous interaction between all stakeholders is essential to monitor, adapt and improve assessment practices and to stimulate the development of a shared mental model. Better understanding of underlying stakeholder perspectives could be an important step in bridging the gap between psychometric and socio-constructivist approaches in WBA.
Collapse
Affiliation(s)
- Laury P J W M de Jonge
- Department of Family Medicine, FHML, Maastricht University, P.O. Box 616, 6200 MD, Maastricht, The Netherlands.
| | - Angelique A Timmerman
- Department of Family Medicine, FHML, Maastricht University, P.O. Box 616, 6200 MD, Maastricht, The Netherlands
| | - Marjan J B Govaerts
- Department of Educational Research and Development, FHML, Maastricht University, Maastricht, The Netherlands
| | - Jean W M Muris
- Department of Family Medicine, FHML, Maastricht University, P.O. Box 616, 6200 MD, Maastricht, The Netherlands
| | - Arno M M Muijtjens
- Department of Educational Research and Development, FHML, Maastricht University, Maastricht, The Netherlands
| | - Anneke W M Kramer
- Department of Family Medicine, Leiden University, Leiden, The Netherlands
| | - Cees P M van der Vleuten
- Department of Educational Research and Development, FHML, Maastricht University, Maastricht, The Netherlands
| |
Collapse
|
19
|
LaDonna KA, Hatala R, Lingard L, Voyer S, Watling C. Staging a performance: learners' perceptions about direct observation during residency. MEDICAL EDUCATION 2017; 51:498-510. [PMID: 28247495 DOI: 10.1111/medu.13232] [Citation(s) in RCA: 87] [Impact Index Per Article: 12.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/05/2016] [Revised: 06/29/2016] [Accepted: 09/27/2016] [Indexed: 05/14/2023]
Abstract
CONTEXT Evidence strongly supports that direct observation is a valid and reliable assessment tool; support for its impact on learning is less compelling, and we know that some learners are ambivalent about being observed. However, learners' perceptions about the impact of direct observation on their learning and professional development remain underexplored. To promote learning, we need to understand what makes direct observation valuable for learners. METHODS Informed by constructivist grounded theory, we interviewed 22 learners about their observation experiences. Data collection and analysis occurred iteratively; themes were identified using constant comparative analysis. RESULTS Direct observation was widely endorsed as an important educational strategy, albeit one that created significant anxiety. Opaque expectations exacerbated participants' discomfort, and participants described that being observed felt like being assessed. Consequently, participants exchanged their 'usual' practice for a 'textbook' approach; alterations to performance generated uncertainty about their role, and raised questions about whether observers saw an authentic portrayal of their knowledge and skill. CONCLUSION An 'observer effect' may partly explain learners' ambivalence about direct observation; being observed seemed to magnify learners' role ambiguity, intensify their tensions around professional development and raise questions about the credibility of feedback. In turn, an observer effect may impact learners' receptivity to feedback and may explain, in part, learners' perceptions that useful feedback is scant. For direct observation to be valuable, educators must be explicit about expectations, and they must be aware that how learners perform in the presence of an observer may not reflect what they do as independent practitioners. To nurture learners' professional development, educators must create a culture of observation-based coaching that is divorced from assessment and is tailored to developing learners' identities as practitioners of both the art and the science of medicine.
Collapse
Affiliation(s)
- Kori A LaDonna
- Centre for Education Research & Innovation, Schulich School of Medicine and Dentistry, Western University, London, Ontario, Canada
| | - Rose Hatala
- Department of Medicine, University of British Columbia, Vancouver, British Columbia, Canada
| | - Lorelei Lingard
- Department of Medicine and Centre for Education Research & Innovation, Schulich School of Medicine and Dentistry, Western University, London, Ontario, Canada
| | - Stephane Voyer
- Department of Medicine, University of British Columbia, Vancouver, British Columbia, Canada
| | - Christopher Watling
- Department of Clinical Neurological Sciences and Centre for Education Research & Innovation, Schulich School of Medicine and Dentistry, Western University, London, Ontario, Canada
| |
Collapse
|
20
|
Berkhout JJ, Teunissen PW, Helmich E, van Exel J, van der Vleuten CPM, Jaarsma DADC. Patterns in clinical students' self-regulated learning behavior: a Q-methodology study. ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2017; 22:105-121. [PMID: 27235123 PMCID: PMC5306423 DOI: 10.1007/s10459-016-9687-4] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/09/2015] [Accepted: 05/14/2016] [Indexed: 05/24/2023]
Abstract
Students feel insufficiently supported in clinical environments to engage in active learning and achieve a high level of self-regulation. As a result clinical learning is highly demanding for students. Because of large differences between students, supervisors may not know how to support them in their learning process. We explored patterns in undergraduate students' self-regulated learning behavior in the clinical environment, to improve tailored supervision, using Q-methodology. Q-methodology uses features of both qualitative and quantitative methods for the systematic investigation of subjective issues by having participants sort statements along a continuum to represent their opinion. We enrolled 74 students between December 2014 and April 2015 and had them characterize their learning behavior by sorting 52 statements about self-regulated learning behavior and explaining their response. The statements used for the sorting were extracted from a previous study. The data was analyzed using by-person factor analysis to identify clusters of individuals with similar sorts of the statements. The resulting factors and qualitative data were used to interpret and describe the patterns that emerged. Five resulting patterns were identified in students' self-regulated learning behavior in the clinical environment, which we labelled: Engaged, Critically opportunistic, Uncertain, Restrained and Effortful. The five patterns varied mostly regarding goals, metacognition, communication, effort, and dependence on external regulation for learning. These discrete patterns in students' self-regulated learning behavior in the clinical environment are part of a complex interaction between student and learning context. The results suggest that developing self-regulated learning behavior might best be supported regarding individual students' needs.
Collapse
Affiliation(s)
- Joris J Berkhout
- Center for Evidence-Based Education, Academic Medical Center (AMC-UvA), University of Amsterdam, Amsterdam, The Netherlands.
| | - Pim W Teunissen
- Department of Educational Development and Research, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, The Netherlands
- Department of Obstetrics and Gynecology, VU University Medical Center, VU University Amsterdam, Amsterdam, The Netherlands
| | - Esther Helmich
- Center for Evidence-Based Education, Academic Medical Center (AMC-UvA), University of Amsterdam, Amsterdam, The Netherlands
| | - Job van Exel
- Institute of Health Policy and Management (BMG), Erasmus University, Rotterdam, The Netherlands
| | - Cees P M van der Vleuten
- Department of Educational Development and Research, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, The Netherlands
| | - Debbie A D C Jaarsma
- Center for Research and Innovation in Medical Education, University Medical Center Groningen, University of Groningen, Groningen, The Netherlands
| |
Collapse
|
21
|
Barrett A, Galvin R, Scherpbier AJJA, Teunissen PW, O'Shaughnessy A, Horgan M. Is the learning value of workplace-based assessment being realised? A qualitative study of trainer and trainee perceptions and experiences. Postgrad Med J 2016; 93:138-142. [DOI: 10.1136/postgradmedj-2015-133917] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2015] [Revised: 06/24/2016] [Accepted: 07/11/2016] [Indexed: 11/04/2022]
|
22
|
Massie J, Ali JM. Workplace-based assessment: a review of user perceptions and strategies to address the identified shortcomings. ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2016; 21:455-73. [PMID: 26003590 DOI: 10.1007/s10459-015-9614-0] [Citation(s) in RCA: 77] [Impact Index Per Article: 9.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/24/2014] [Accepted: 05/13/2015] [Indexed: 05/12/2023]
Abstract
Workplace based assessments (WBAs) are now commonplace in postgraduate medical training. User acceptability and engagement is essential to the success of any medical education innovation. To this end, possessing an insight into trainee and trainer perceptions towards WBAs will help identify the major problems, permitting strategies to be introduced to improve WBA implementation. A review of literature was performed to identify studies examining trainee and trainer perceptions towards WBAs. Studies were excluded if non-English or sampling a non-medical/dental population. The identified literature was synthesised for the purpose of this critical narrative review. It is clear that there is widespread negativity towards WBAs in the workplace. This has negatively impacted on the effectiveness of WBA tools as learning aids. This negativity exists in trainees but also to an extent in their trainers. Insight gained from the literature reveals three dominant problems with WBA implementation: poor understanding as to the purpose of WBAs; insufficient time available for undertaking these assessments; and inadequate training of trainers. Approaches to addressing these three problems with WBA implementation are discussed. It is likely that a variety of solutions will be required. The prevalence of negativity towards WBAs is substantial in both trainees and trainers, eroding the effectiveness of learning that is consequent upon them. The educational community must now listen to the concerns being raised by the users and consider the range of strategies being proposed to improve the experiences of trainees, and their trainers.
Collapse
Affiliation(s)
- Jonathan Massie
- School of Clinical Medicine, University of Cambridge, Cambridge, UK
| | - Jason M Ali
- Department of Surgery, University of Cambridge, BOX 202, Addenbrookes Hospital, Cambridge, CB2 0QQ, UK.
| |
Collapse
|
23
|
Fokkema JPI. Innovating the practice of medical speciality training. PERSPECTIVES ON MEDICAL EDUCATION 2016; 5:48-50. [PMID: 26754312 PMCID: PMC4754224 DOI: 10.1007/s40037-015-0245-1] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/02/2023]
Abstract
Educational innovations are being introduced into medical speciality training. But how do people who participate in medical speciality training (residents, consultants, programme directors) deal with these innovations? And what effects do educational innovations have according to these people?By addressing these questions, this thesis contributes to the knowledge about the challenging process of innovating medical speciality training.
Collapse
|
24
|
Lefroy J, Watling C, Teunissen PW, Brand P. Guidelines: the do's, don'ts and don't knows of feedback for clinical education. PERSPECTIVES ON MEDICAL EDUCATION 2015; 4:284-299. [PMID: 26621488 PMCID: PMC4673072 DOI: 10.1007/s40037-015-0231-7] [Citation(s) in RCA: 182] [Impact Index Per Article: 20.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/09/2023]
Abstract
INTRODUCTION The guidelines offered in this paper aim to amalgamate the literature on formative feedback into practical Do's, Don'ts and Don't Knows for individual clinical supervisors and for the institutions that support clinical learning. METHODS The authors built consensus by an iterative process. Do's and Don'ts were proposed based on authors' individual teaching experience and awareness of the literature, and the amalgamated set of guidelines were then refined by all authors and the evidence was summarized for each guideline. Don't Knows were identified as being important questions to this international group of educators which if answered would change practice. The criteria for inclusion of evidence for these guidelines were not those of a systematic review, so indicators of strength of these recommendations were developed which combine the evidence with the authors' consensus. RESULTS A set of 32 Do and Don't guidelines with the important Don't Knows was compiled along with a summary of the evidence for each. These are divided into guidelines for the individual clinical supervisor giving feedback to their trainee (recommendations about both the process and the content of feedback) and guidelines for the learning culture (what elements of learning culture support the exchange of meaningful feedback, and what elements constrain it?) CONCLUSION Feedback is not easy to get right, but it is essential to learning in medicine, and there is a wealth of evidence supporting the Do's and warning against the Don'ts. Further research into the critical Don't Knows of feedback is required. A new definition is offered: Helpful feedback is a supportive conversation that clarifies the trainee's awareness of their developing competencies, enhances their self-efficacy for making progress, challenges them to set objectives for improvement, and facilitates their development of strategies to enable that improvement to occur.
Collapse
Affiliation(s)
- Janet Lefroy
- Keele University School of Medicine, Clinical Education Centre RSUH, ST4 6QG, Staffordshire, UK.
| | - Chris Watling
- Schulich School of Medicine and Dentistry, Western University, Ontario, Canada
| | - Pim W Teunissen
- Maastricht University and VU University Medical Center, Amsterdam, The Netherlands
| | - Paul Brand
- Isala Klinieken, Zwolle, The Netherlands
| |
Collapse
|
25
|
Kogan JR, Conforti LN, Bernabeo E, Iobst W, Holmboe E. How faculty members experience workplace-based assessment rater training: a qualitative study. MEDICAL EDUCATION 2015; 49:692-708. [PMID: 26077217 DOI: 10.1111/medu.12733] [Citation(s) in RCA: 49] [Impact Index Per Article: 5.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/21/2014] [Revised: 11/13/2014] [Accepted: 02/11/2015] [Indexed: 05/09/2023]
Abstract
CONTEXT Direct observation of clinical skills is a common approach in workplace-based assessment (WBA). Despite widespread use of the mini-clinical evaluation exercise (mini-CEX), faculty development efforts are typically required to improve assessment quality. Little consensus exists regarding the most effective training methods, and few studies explore faculty members' reactions to rater training. OBJECTIVES This study was conducted to qualitatively explore the experiences of faculty staff with two rater training approaches - performance dimension training (PDT) and a modified approach to frame of reference training (FoRT) - to elucidate how such faculty development can be optimally designed. METHODS In a qualitative study of a multifaceted intervention using complex intervention principles, 45 out-patient resident faculty preceptors from 26 US internal medicine residency programmes participated in a rater training faculty development programme. All participants were interviewed individually and in focus groups during and after the programme to elicit how the training influenced their approach to assessment. A constructivist grounded theory approach was used to analyse the data. RESULTS Many participants perceived that rater training positively influenced their approach to direct observation and feedback, their ability to use entrustment as the standard for assessment, and their own clinical skills. However, barriers to implementation and change included: (i) a preference for holistic assessment over frameworks; (ii) challenges in defining competence; (iii) difficulty in changing one's approach to assessment, and (iv) concerns about institutional culture and buy-in. CONCLUSIONS Rater training using PDT and a modified approach to FoRT can provide faculty staff with assessment skills that are congruent with principles of criterion-referenced assessment and entrustment, and foundational principles of competency-based education, while providing them with opportunities to reflect on their own clinical skills. However, multiple challenges to incorporating new forms of training exist. Ongoing efforts to improve WBA are needed to address institutional and cultural contexts, and systems of care delivery.
Collapse
Affiliation(s)
- Jennifer R Kogan
- Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Lisa N Conforti
- Milestones Development and Evaluation, Accreditation Council of Graduate Medical Education, Chicago, Illinois, USA
| | - Elizabeth Bernabeo
- Evaluation Research and Development, American Board of Internal Medicine, Philadelphia, Pennsylvania, USA
| | - William Iobst
- Academic and Clinical Affairs, Commonwealth Medical College, Scranton, Pennsylvania, USA
| | - Eric Holmboe
- Milestones Development and Evaluation, Accreditation Council of Graduate Medical Education, Chicago, Illinois, USA
| |
Collapse
|
26
|
Fokkema JPI. In Reply to Ali. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2015; 90:839. [PMID: 26414049 DOI: 10.1097/acm.0000000000000762] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/05/2023]
|
27
|
Ali JM. Discordance Persists Between Trainers' and Trainees' Perceptions of Workplace-Based Assessment. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2015; 90:839. [PMID: 26414048 DOI: 10.1097/acm.0000000000000746] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/05/2023]
Affiliation(s)
- Jason M Ali
- Academic clinical fellow, Department of Surgery, University of Cambridge, Cambridge, United Kingdom;
| |
Collapse
|