1
|
van Keulen SG, de Raad T, Raymakers-Janssen P, Ten Cate O, Hennus MP. Exploring Interprofessional Development of Entrustable Professional Activities For Pediatric Intensive Care Fellows: A Proof-of-Concept Study. TEACHING AND LEARNING IN MEDICINE 2024; 36:154-162. [PMID: 37071751 DOI: 10.1080/10401334.2023.2200760] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/14/2022] [Accepted: 03/13/2023] [Indexed: 06/19/2023]
Abstract
Phenomenon: Entrustable professional activities (EPAs) delineate major professional activities that an individual in a given specialty must be "entrusted" to perform, ultimately without supervision, to provide quality patient care. Until now, most EPA frameworks have been developed by professionals within the same specialty. As safe, effective, and sustainable health care ultimately depends on interprofessional collaboration, we hypothesized that members of interprofessional teams might have clear and possibly additional insight into which activities are essential to the professional work of a medical specialist. Approach: We recently employed a national modified Delphi study to develop and validate a set of EPAs for Dutch pediatric intensive care fellows. In this proof-of-concept study, we explored what pediatric intensive care physicians' non-physician team members (physician assistants, nurse practitioners, and nurses) constitute as essential professional activities for PICU physicians and how they regarded the newly developed set of nine EPAs. We compared their judgments with the PICU physicians' opinions. Findings: This study shows that non-physician team members share a mental model with physicians about which EPAs are indispensable for pediatric intensive care physicians. Despite this agreement however, descriptions of EPAs are not always clear for non-physician team members who have to work with them on a daily basis. Insights: Ambiguity as to what an EPA entails when qualifying a trainee can have implications for patient safety and trainees themselves. Input from non-physician team members may add to the clarity of EPA descriptions. This finding supports the involvement of non-physician team members in the developmental process of EPAs for (sub)specialty training programs.
Collapse
Affiliation(s)
- Sabrina G van Keulen
- Wilhelmina Children's Hospital, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Timo de Raad
- Pediatric Intensive Care, Wilhelmina Children's Hospital, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Paulien Raymakers-Janssen
- Pediatric Intensive Care, Wilhelmina Children's Hospital, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Olle Ten Cate
- Utrecht Center for Research and Development of Health Professions Education, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Marije P Hennus
- Pediatric Intensive Care, Wilhelmina Children's Hospital, University Medical Center Utrecht, Utrecht, the Netherlands
| |
Collapse
|
2
|
Rawashdeh MA, Brennan PC. Reducing ' probably benign ' assessments in normal mammograms: The role of radiologist experience. Eur J Radiol Open 2023; 10:100498. [PMID: 37359179 PMCID: PMC10285087 DOI: 10.1016/j.ejro.2023.100498] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2023] [Revised: 06/07/2023] [Accepted: 06/09/2023] [Indexed: 06/28/2023] Open
Abstract
Rationale and objectives to investigate the relationship between radiologists' experience in reporting mammograms, their caseloads, and the classification of category '3' or 'Probably Benign' on normal mammograms. Materials and Methods A total of 92 board-certified radiologists participated. Self-reported parameters related to experience, including age, years since qualifying as a radiologist, years of experience reading mammograms, number of mammograms read per year, and hours spent reading mammograms per week, were documented. To assess the radiologists' accuracy, "Probably Benign fractions" was calculated by dividing the number of "Probably Benign findings" given by each radiologist in the normal cases by the total number of normal cases Probably Benign fractions were correlated with various factors, such as the radiologists' experience. Results The results of the statistical analysis revealed a significant negative correlation between radiologist experience and 'Probably Benign' fractions for normal images. Specifically, for normal cases, the number of mammograms read per year (r = -0.29, P = 0.006) and the number of mammograms read over the radiologist's lifetime (r = -0.21, P = 0.049) were both negatively correlated with 'Probably Benign' fractions. Conclusion The results indicate that a relationship exists between increased reading volumes and reduced assessments of 'Probably Benign' in normal mammograms. The implications of these findings extend to the effectiveness of screening programs and the recall rates.
Collapse
Affiliation(s)
- Mohammad A. Rawashdeh
- Faculty of Health Sciences, Gulf Medical University, Ajman, United Arab Emirates
- Faculty of Applied Medical Sciences, Jordan University of Science and Technology, Irbid 222110, Jordan
| | - Patrick C. Brennan
- Medical Image Optimisation and Perception Group (MIOPeG), Sydney School of Health Sciences, Faculty of Medicine and Health, The University of Sydney, Sydney, New South Wales, Australia
| |
Collapse
|
3
|
Allen L, Hall AK, Braund H, Chaplin T. Catalytic effect of multisource feedback for trauma team captains: a mixed-methods prospective study. BMJ Open 2023; 13:e068732. [PMID: 37221034 DOI: 10.1136/bmjopen-2022-068732] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 05/25/2023] Open
Abstract
OBJECTIVES To evaluate the impact and feasibility of multisource feedback compared with traditional feedback for trauma team captains (TTCs). DESIGN A mixed-methods, non-randomised prospective study. SETTING A level one trauma centre in Ontario, Canada. PARTICIPANTS Postgraduate medical residents in emergency medicine and general surgery participating as TTCs. Selection was based on a convenience sampling method. INTERVENTION Postgraduate medical residents participating as TTCs received either multisource feedback or standard feedback following trauma cases. MAIN OUTCOME MEASURES TTCs completed questionnaires designed to measure the self-reported intention to change practice (catalytic effect), immediately following a trauma case and 3 weeks later. Secondary outcomes included measures of perceived benefit, acceptability, and feasibility from TTCs and other trauma team members. RESULTS Data were collected following 24 trauma team activations: TTCs from 12 activations received multisource feedback and 12 received standard feedback. The self-reported intention for practice change was not significantly different between groups initially (4.0 vs 4.0, p=0.57) and at 3 weeks (4.0 vs 3.0, p=0.25). Multisource feedback was perceived to be helpful and superior to the existing feedback process. Feasibility was identified as a challenge. CONCLUSIONS The self-reported intention for practice change was no different for TTCs who received multisource feedback and those who received standard feedback. Multisource feedback was favourably received by trauma team members, and TTCs perceived multisource feedback as useful for their development.
Collapse
Affiliation(s)
- Leah Allen
- Department of Emergency Medicine, Queen's University, Kingston, Ontario, Canada
| | - Andrew K Hall
- Department of Emergency Medicine, University of Ottawa Faculty of Medicine, Ottawa, Ontario, Canada
| | - Heather Braund
- Office of Professional Development and Educational Scholarship, Queen's University Faculty of Health Sciences, Kingston, Ontario, Canada
| | - Timothy Chaplin
- Department of Emergency Medicine, Queen's University, Kingston, Ontario, Canada
| |
Collapse
|
4
|
Eijkelboom MCL, de Kleijn RAM, van Diemen WJM, Maljaars CDN, van der Schaaf MF, Frenkel J. Patients as Feedback Providers: Exploring Medical Students' Credibility Judgments. PERSPECTIVES ON MEDICAL EDUCATION 2023; 12:129-140. [PMID: 37064270 PMCID: PMC10103723 DOI: 10.5334/pme.842] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/09/2022] [Accepted: 03/07/2023] [Indexed: 05/05/2023]
Abstract
Introduction Patient feedback is becoming ever more important in medical education. Whether students engage with feedback is partly determined by how credible they think the feedback provider is. Despite its importance for feedback engagement, little is known about how medical students judge the credibility of patients. The purpose of this study was therefore to explore how medical students make credibility judgments regarding patients as feedback providers. Methods This qualitative study builds upon McCroskey's conceptualization of credibility as a three-dimensional construct comprising: competence, trustworthiness, and goodwill. Since credibility judgments are shaped by the context, we studied students' credibility judgments in both a clinical and non-clinical context. Medical students were interviewed after receiving feedback from patients. Interviews were analyzed through template and causal network analysis. Results Students based their credibility judgments of patients on multiple interacting arguments comprising all three dimensions of credibility. In estimating a patient's credibility, students reasoned about aspects of the patient's competence, trustworthiness, and goodwill. In both contexts students perceived elements of an educational alliance between themselves and patients, which could increase credibility. Yet, in the clinical context students reasoned that therapeutic goals of the relationship with patients might impede educational goals of the feedback interaction, which lowered credibility. Discussion Students' credibility judgments of patients were a weighing of multiple sometimes conflicting factors, within the context of relationships and their associated goals. Future research should explore how goals and roles can be discussed between students and patients to set the stage for open feedback conversations.
Collapse
Affiliation(s)
- M. C. L. Eijkelboom
- Faculty of Medicine, Utrecht University, the Netherlands
- Department of Pediatrics, University Medical Center Utrecht, the Netherlands
| | - R. A. M. de Kleijn
- Utrecht Center for Research and Development of Health Professions Education, University Medical Center Utrecht, the Netherlands
| | - W. J. M. van Diemen
- Utrecht Center for Research and Development of Health Professions Education, University Medical Center Utrecht, the Netherlands
| | | | - M. F. van der Schaaf
- Utrecht Center for Research and Development of Health Professions Education, University Medical Center Utrecht, the Netherlands
| | - J. Frenkel
- Department of Pediatrics, University Medical Center Utrecht, the Netherlands
| |
Collapse
|
5
|
Khan YS, Khoodoruth MAS, Ghaffar A, Al Khal A, Alabdullah M. The Impact of Multisource Feedback on Continuing Medical Education, Clinical Performance and Patient Experience: Innovation in a Child and Adolescent Mental Health Service. JOURNAL OF CME 2023; 12:2202834. [PMID: 37123200 PMCID: PMC10142306 DOI: 10.1080/28338073.2023.2202834] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 05/02/2023]
Abstract
This paper reiterates the importance of the role of multisource feedback (MSF) in continuing medical education/continuing professional development (CME/CPD) and its impact on doctors' performance and patient experience globally. It summarises a unique initiative of robust utilisation of internationally recognised multisource feedback tools in an outpatient child and adolescent mental health service (CAMHS) in Qatar. The process involved the effective adoption and administering of the General Medical Council's (GMC) self-assessment questionnaire (SQ), patient questionnaire (PQ), and colleague questionnaire (CQ) followed by the successful incorporation of these tools in CME/CPD. The original version of the PQ questionnaire and the instructions to the patient document were translated into Arabic through the blind back-translation technique. This initiative of introducing gold-standard MSF tools and processes into clinical practice, among other quality-improvement projects, has contributed to the improvement of service standards and doctors' clinical practice. Patient satisfaction was measured through the annual patient experience analysis using the Experience of Service Questionnaire (ESQ) whereas changes in doctors' performance were evaluated by comparing annual appraisal scores before and after implementation of this initiative. We have demonstrated that when MSF is obtained impartially and transparently using recognised and valid tools, it can improve patient experience and enhance doctors' performance.
Collapse
Affiliation(s)
- Yasser Saeed Khan
- Mental Health Service, Hamad Medical Corporation, Doha, Qatar
- CONTACT Yasser Saeed Khan Mental Health Service,Hamad Medical Corporation, Doha, Qatar P.O Box 3050
| | - Mohamed Adil Shah Khoodoruth
- Mental Health Service, Hamad Medical Corporation, Doha, Qatar
- Division of Genomics and Precision Medicine, College of Health and Life Sciences, Hamad Bin Khalifa University, Doha, Qatar
| | - Adeel Ghaffar
- Graduat
e Medical Education, Hamad Medical Corporation, Doha, Qatar
| | | | - Majid Alabdullah
- Mental Health Service, Hamad Medical Corporation, Doha, Qatar
- College of Medicine, Qatar University, Doha, Qatar
| |
Collapse
|
6
|
Webber K, Selman R. UK palliative medicine trainees and multisource communication skills feedback: an educational tool? BMJ Support Palliat Care 2022; 12:e485-e488. [DOI: 10.1136/bmjspcare-2019-002133] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2019] [Revised: 01/28/2020] [Accepted: 03/09/2020] [Indexed: 11/04/2022]
Abstract
BackgroundMultisource feedback provides ratings of a trainee doctor’s performance from a range of assessors and enables 360 degree feedback on communication skills and team working behaviours. It is a tool used throughout palliative medicine training in the UK. There are limited data on the value of multisource feedback from a palliative medicine trainee perspective.AimTo study the views of palliative medicine trainees regarding multisource feedback as an educational tool to develop communication skills.DesignA multimodal study encompassing a focus group and questionnaire mailed to all deanery palliative doctors.Setting/participantsAll palliative medicine trainees within a UK training deanery.ResultsOver half of responding trainees thought multisource feedback had little or no impact on their clinical practice. Improvements in delivery of multisource feedback to maximise learning were identified, including skilled feedback and facilitation by educational supervisors.ConclusionsDespite multisource feedback currently having limited benefits, a number of recommendations are suggested to improve this.
Collapse
|
7
|
Roy M, Lockyer J, Touchie C. Family Physician Quality Improvement Plans: A Realist Inquiry Into What Works, for Whom, Under What Circumstances. THE JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS 2022; 43:155-163. [PMID: 37638679 DOI: 10.1097/ceh.0000000000000454] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/29/2023]
Abstract
INTRODUCTION Evaluation of quality improvement programs shows variable impact on physician performance often neglecting to examine how implementation varies across contexts and mechanisms that affect uptake. Realist evaluation enables the generation, refinement, and testing theories of change by unpacking what works for whom under what circumstances and why. This study used realist methods to explore relationships between outcomes, mechanisms (resources and reasoning), and context factors of a national multisource feedback (MSF) program. METHODS Linked data for 50 physicians were examined to determine relationships between action plan completion status (outcomes), MSF ratings, MSF comments and prescribing data (resource mechanisms), a report summarizing the conversation between a facilitator and physician (reasoning mechanism), and practice risk factors (context). Working backward from outcomes enabled exploration of similarities and differences in mechanisms and context. RESULTS The derived model showed that the completion status of plans was influenced by interaction of resource and reasoning mechanisms with context mediating the relationships. Two patterns were emerged. Physicians who implemented all their plans within six months received feedback with consistent messaging, reviewed data ahead of facilitation, coconstructed plan(s) with the facilitator, and had fewer risks to competence (dyscompetence). Physicians who were unable to implement any plans had data with fewer repeated messages and did not incorporate these into plans, had difficult plans, or needed to involve others and were physician-led, and were at higher risk for dyscompetence. DISCUSSION Evaluation of quality improvement initiatives should examine program outcomes taking into consideration the interplay of resources, reasoning, and risk factors for dyscompetence.
Collapse
Affiliation(s)
- Marguerite Roy
- Roy: Adjunct Professor, Department of Innovation in Medical Education, University of Ottawa, Ottawa, Canada. Lockyer: Professor, Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, Canada. Touchie: Professor of Medicine, Department of Medicine, University of Ottawa and the Ottawa Hospital, Ottawa, Canada, and Department of Innovation in Medical Education, Medical Council of Canada, Ottawa, Canada
| | | | | |
Collapse
|
8
|
Hennel EK, Trachsel A, Subotic U, Lörwald AC, Harendza S, Huwendiek S. How does multisource feedback influence residency training? A qualitative case study. MEDICAL EDUCATION 2022; 56:660-669. [PMID: 35263461 PMCID: PMC9314722 DOI: 10.1111/medu.14798] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/26/2021] [Revised: 01/20/2022] [Accepted: 02/22/2022] [Indexed: 06/14/2023]
Abstract
INTRODUCTION Multisource feedback (MSF), also called 360-degree assessment, is one form of assessment used in postgraduate training. However, there is an ongoing discussion on its value, because the factors that influence the impact of MSF and the main impact of MSF are not fully understood. In this study, we investigated both the influencing factors and the impact of MSF on residency training. METHODS We conducted a qualitative case study within the boundaries of the residency training for paediatricians and paediatric surgeons at a University Hospital. We collected data from seven focus group interviews with stakeholders of MSF (residents, raters and supervisors). By performing a reflexive thematic analysis, we extracted the influencing factors and the impact of MSF. RESULTS We found seven influencing factors: MSF is facilitated by the announcement of a clear goal of MSF, the training of raters on the MSF instrument, a longitudinal approach of observation, timing not too early and not too late during the rotation, narrative comments as part of the ratings, the residents' self-assessment and a supervisor from the same department. We found three themes on the impact of MSF: MSF supports the professional development of residents, enhances interprofessional teamwork and increases the raters' commitment to the training of residents. CONCLUSION This study illuminates the influencing factors and impact of MSF on residency training. We offer novel recommendations on the continuity of observation, the timing during rotations and the role of the supervisor. Moreover, by discussing our results through the lens of identity formation theory, this work advances our conceptual understanding of MSF. We propose identity formation theory as a framework for future research on MSF to leverage the potential of MSF in residency training.
Collapse
Affiliation(s)
- Eva K. Hennel
- Department for Assessment and Evaluation (AAE), Institute for Medical EducationUniversity of BernBernSwitzerland
| | - Andrea Trachsel
- Department for Assessment and Evaluation (AAE), Institute for Medical EducationUniversity of BernBernSwitzerland
| | | | - Andrea C. Lörwald
- Department for Assessment and Evaluation (AAE), Institute for Medical EducationUniversity of BernBernSwitzerland
| | - Sigrid Harendza
- Department of Internal MedicineUniversity Medical Center Hamburg‐EppendorfHamburgGermany
| | - Sören Huwendiek
- Department for Assessment and Evaluation (AAE), Institute for Medical EducationUniversity of BernBernSwitzerland
| |
Collapse
|
9
|
Bhat C, LaDonna KA, Dewhirst S, Halman S, Scowcroft K, Bhat S, Cheung WJ. Unobserved Observers: Nurses' Perspectives About Sharing Feedback on the Performance of Resident Physicians. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2022; 97:271-277. [PMID: 34647919 DOI: 10.1097/acm.0000000000004450] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
PURPOSE Postgraduate training programs are incorporating feedback from registered nurses (RNs) to facilitate holistic assessments of resident performance. RNs are a potentially rich source of feedback because they often observe trainees during clinical encounters when physician supervisors are not present. However, RN perspectives about sharing feedback have not been deeply explored. This study investigated RN perspectives about providing feedback and explored the facilitators and barriers influencing their engagement. METHOD Constructivist grounded theory methodology was used in interviewing 11 emergency medicine and 8 internal medicine RNs at 2 campuses of a tertiary care academic medical center in Ontario, Canada, between July 2019 and March 2020. Interviews explored RN experiences working with and observing residents in clinical practice. Data collection and analysis were conducted iteratively. Themes were identified using constant comparative analysis. RESULTS RNs felt they could observe authentic day-to-day behaviors of residents often unwitnessed by supervising physicians and offer unique feedback related to patient advocacy, communication, leadership, collaboration, and professionalism. Despite a strong desire to contribute to resident education, RNs were apprehensive about sharing feedback and reported barriers related to hierarchy, power differentials, and a fear of overstepping professional boundaries. Although infrequent, a key stimulus that enabled RNs to feel safe in sharing feedback was an invitation from the supervising physician to provide input. CONCLUSIONS Perceived hierarchy in academic medicine is a critical barrier to engaging RNs in feedback for residents. Accessing RN feedback on authentic resident behaviors requires dismantling the negative effects of hierarchy and fostering a collaborative interprofessional working environment. A critical step toward this goal may require supervising physicians to model feedback-seeking behavior by inviting RNs to share feedback. Until a workplace culture is established that validates nurses' input and creates safe opportunities for them to contribute to resident education, the voices of nurses will remain unheard.
Collapse
Affiliation(s)
- Chirag Bhat
- C. Bhat is a resident physician, Department of Emergency Medicine, University of Ottawa, Ottawa, Ontario, Canada; ORCID: https://orcid.org/0000-0003-3198-6450
| | - Kori A LaDonna
- K.A. LaDonna is assistant professor, Department of Innovation in Medical Education, University of Ottawa, Ottawa, Ontario, Canada
| | - Sebastian Dewhirst
- S. Dewhirst is a lecturer, Department of Emergency Medicine, University of Ottawa, Ottawa, Ontario, Canada; ORCID: https://orcid.org/0000-0003-1996-6692
| | - Samantha Halman
- S. Halman is assistant professor, Department of Medicine, University of Ottawa and the Ottawa Hospital, Ottawa, Ontario, Canada; ORCID: http://orcid.org/0000-0002-5474-9696
| | - Katherine Scowcroft
- K. Scowcroft is a research assistant, Department of Innovation in Medical Education, University of Ottawa, Ottawa, Ontario, Canada
| | - Silke Bhat
- S. Bhat is a registered nurse, Department of Emergency Medicine, the Ottawa Hospital, Ottawa, Ontario, Canada
| | - Warren J Cheung
- W.J. Cheung is associate professor, Department of Emergency Medicine, University of Ottawa, Ottawa, Ontario, Canada; ORCID: https://orcid.org/0000-0002-2730-8190
| |
Collapse
|
10
|
Daniel SJ, Bouchard MJ, Tremblay M. Rethinking Our Annual Congress-Meeting the Needs of Specialist Physicians by Partnering With Provincial Simulation Centers. THE JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS 2022; 42:e83-e87. [PMID: 34609357 PMCID: PMC8876424 DOI: 10.1097/ceh.0000000000000381] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Canada's maintenance of certification programs for physicians has evolved to emphasize assessment activities. Our organization recognized the importance of offering more practice assessment opportunities to our members to enhance their practice and help them comply with a regulation from our provincial professional body related to ongoing continuing education. This led us to rethink our annual congress and enrich the program with a curriculum of interdisciplinary simulation sessions tailored to meet the needs of a broad audience of specialists. Our challenges are similar to those of many national specialty societies having limited access to simulation facilities, instructors, and simulation teams that can cover the breadth and scope of perceived and unperceived simulation needs for their specialty. Our innovative solution was to partner with local experts to develop 22 simulation sessions over the past three years. The response was very positive, drawing 867 participants. Over 95% of participants either agreed or strongly agreed that their simulation session (1) met their learning objectives, (2) was relevant for their practice, and (3) encouraged them to modify their practice. Narrative comments from a survey sent to the 2018 participants four months after their activity indicated several self-reported changes in their practice or patient outcomes. We were able to centralize offers from organizations that had previously worked in silo to develop simulation sessions meeting the needs of our members. Proposing simulation sessions allowed our organization to establish long-term partnerships and to expend our "educational toolbox" to address skill gaps not usually addressed during annual meetings.
Collapse
|
11
|
Ahmad CN, Sethi A, Khan RA. Impact of implementing multisource feedback on behaviors of young doctors. Pak J Med Sci 2021; 37:1953-1958. [PMID: 34912425 PMCID: PMC8613060 DOI: 10.12669/pjms.37.7.4155] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2021] [Revised: 05/08/2021] [Accepted: 06/25/2021] [Indexed: 11/17/2022] Open
Abstract
Objectives: Multisource feedback (MSF) is a workplace-based assessment tool that offers 360-degree evaluation of the trainee doctor. Little is known about its receptiveness among stakeholders in Pakistan. This study explores house officers’ perceptions regarding MSF since its implementation in Eye Unit-II, Institute of Ophthalmology, King Edward Medical University/ Mayo Hospital, Lahore. Methods: A qualitative case study was conducted from July 2019 to February 2020 in Eye Unit II. A purposive (maximum variation) sample of 12 house surgeons was taken. Two focus group discussions were conducted. Data were transcribed and analyzed thematically. Results: The study identified the impact of MSF on house surgeons. Most participants reported positive experiences. The feedback they received increased their motivation, management skills and team working. A number of factors affecting the receptiveness of MSF were also identified which mainly included characteristics of raters and emotional response to MSF. Conclusion: Multisource Feedback is a useful tool for feedback that impacts the young doctors in many ways. It contributes to increasing their sense of responsibility, management skills and self-directed learning. The improvement in individual abilities and teamwork also helped in improving patient care.
Collapse
Affiliation(s)
- Ch Nasir Ahmad
- Dr. Ch Nasir Ahmad, MBBS, FCPS, FICO, Fellowship in vitreo-retina, MME. Department of Ophthalmology Unit-II, King Edward Medical University, Lahore, Pakistan
| | - Ahsan Sethi
- Dr. Ahsan Sethi, BDS MPH MMEd FHEA MAcadMEd FDTFEd PhD. Department of Public Health, College of Health Sciences, QU Health, Qatar University, Doha, Qatar
| | - Rehan Ahmed Khan
- Prof. Dr. Rehan Ahmed Khan, MBBS, FCPS, FRCS, JMHPE, MSc-HPE, MHPE. Islamic International Medical College, Riphah International University, Islamabad, Pakistan
| |
Collapse
|
12
|
Sureda E, Chacón-Moscoso S, Sanduvete-Chaves S, Sesé A. A Training Intervention through a 360° Multisource Feedback Model. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2021; 18:ijerph18179137. [PMID: 34501727 PMCID: PMC8431571 DOI: 10.3390/ijerph18179137] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/26/2021] [Revised: 08/24/2021] [Accepted: 08/27/2021] [Indexed: 11/29/2022]
Abstract
Physicians and other health sciences professionals need continuous training, not only in technical aspects of their activity but also in nontechnical, transversal competencies with a cost-efficient impact on the proper functioning of healthcare. The objective of this paper is to analyze the behavioral change among health professionals at a large public hospital following a training intervention on a set of core nontechnical competencies: Teamwork, Adaptability-Flexibility, Commitment-Engagement, Results Orientation, and Leadership Skills for Supervisors. The 360° Multisource Feedback (MSF) model was applied using three sources of information: supervisors, co-workers, and the workers themselves (self-assessment). A quasi-experimental pretest–post-test single-group design with two points in time was utilized. The training intervention improved the scores of only one of the trained competencies—the “Results Orientation” competency—although the scores were slightly inflated. Moreover, significant discrepancies were detected between the three sources, with supervisors awarding the highest scores. The magnitude of behavioral change was related to certain sociodemographic and organizational variables. The study was not immune to the ceiling effect, despite control measures aimed at avoiding it. The empirical evidence suggests that the 360° MSF model must be maintained over time to enhance and reinforce an evaluation culture for better patient care.
Collapse
Affiliation(s)
- Elena Sureda
- Department of Psychology, University of Balearic Islands, 07122 Palma, Spain;
| | - Salvador Chacón-Moscoso
- Experimental Psychology Department, Universidad de Sevilla, 41018 Sevilla, Spain;
- Department of Psychology, Universidad Autónoma de Chile, Santiago 7500138, Chile
- Correspondence: (S.C.-M.); (A.S.)
| | | | - Albert Sesé
- Department of Psychology, University of Balearic Islands, 07122 Palma, Spain;
- Balearic Islands Health Research Institute (IdISBa), 07120 Palma, Spain
- Correspondence: (S.C.-M.); (A.S.)
| |
Collapse
|
13
|
Lewis JJ, Balaji L, Grossestreuer AV, Ullman E, Rosen C, Dubosh NM. Correlation of attending and patient assessment of resident communication skills in the emergency department. AEM EDUCATION AND TRAINING 2021; 5:e10629. [PMID: 34485802 PMCID: PMC8391985 DOI: 10.1002/aet2.10629] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/10/2020] [Revised: 04/12/2021] [Accepted: 05/05/2021] [Indexed: 06/13/2023]
Abstract
BACKGROUND Communication and interpersonal skills are one of the Accreditation Council for Graduate Medical Education's six core competencies. Validated methods for assessing these among trainees are lacking. Educators have developed various communication assessment tools from both the supervising attending and the patient perspectives. How these different assessment methods and tools compare with each other remains unknown. The goal of this study was to determine the degree of agreement between attending and patient assessment of resident communication skills. METHODS This was a retrospective study of emergency medicine (EM) residents at an academic medical center. From July 2017 to June 2018, residents were assessed on communication skills during their emergency department shifts by both their supervising attending physicians and their patients. The attendings rated residents' communication skills with patients, colleagues, and nursing/ancillary staff using a 1 to 5 Likert scale. Patients completed the modified Communication Assessment Tool (CAT), a 14-item questionnaire based on a 1 to 5 Likert scale. Mean attending ratings and patient CAT scores were calculated for each resident. Means were divided into tertiles due to nonparametric distribution of scores. Agreement between attending and patient ratings of residents were measured using Cohen's kappa for each attending evaluation question. Scores were weighted to assign adjacent tertiles partial agreement. RESULTS During the study period, 1,097 attending evaluations and 952 patient evaluations were completed for 26 residents. Attending scores and CAT scores of the residents showed slight to fair agreement in the following three domains: patient communication (κ = 0.21), communication with colleagues (κ = 0.21), and communication with nursing/ancillary staff (κ = 0.26). CONCLUSIONS Attending and patient ratings of EM residents' communication skills show slight to fair agreement. The use of different types of raters may be beneficial in fully assessing trainees' communication skills.
Collapse
Affiliation(s)
- Jason J. Lewis
- Department of Emergency MedicineBeth Israel Deaconess Medical Center/Harvard Affiliated Emergency Medicine ResidencyBostonMassachusettsUSA
| | - Lakshman Balaji
- Department of Emergency MedicineBeth Israel Deaconess Medical CenterBostonMassachusettsUSA
| | - Anne V. Grossestreuer
- Department of Emergency MedicineBeth Israel Deaconess Medical CenterBostonMassachusettsUSA
| | - Edward Ullman
- Department of Emergency MedicineBeth Israel Deaconess Medical Center/Harvard Affiliated Emergency Medicine ResidencyBostonMassachusettsUSA
| | - Carlo Rosen
- Department of Emergency MedicineBeth Israel Deaconess Medical Center/Harvard Affiliated Emergency Medicine ResidencyBostonMassachusettsUSA
| | - Nicole M. Dubosh
- Department of Emergency MedicineBeth Israel Deaconess Medical Center/Harvard Affiliated Emergency Medicine ResidencyBostonMassachusettsUSA
| |
Collapse
|
14
|
Holm EA, Al-Bayati SJL, Barfod TS, Lembeck MA, Pedersen H, Ramberg E, Klemmensen ÅK, Sorensen JL. Feasibility, quality and validity of narrative multisource feedback in postgraduate training: a mixed-method study. BMJ Open 2021; 11:e047019. [PMID: 34321296 PMCID: PMC8319975 DOI: 10.1136/bmjopen-2020-047019] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/03/2022] Open
Abstract
OBJECTIVES To examine a narrative multisource feedback (MSF) instrument concerning feasibility, quality of narrative comments, perceptions of users (face validity), consequential validity, discriminating capacity and number of assessors needed. DESIGN Qualitative text analysis supplemented by quantitative descriptive analysis. SETTING Internal Medicine Departments in Zealand, Denmark. PARTICIPANTS 48 postgraduate trainees in internal medicine specialties, 1 clinical supervisor for each trainee and 376 feedback givers (respondents). INTERVENTION This study examines the use of an electronic, purely narrative MSF instrument. After the MSF process, the trainee and the supervisor answered a postquestionnaire concerning their perception of the process. The authors coded the comments in the MSF reports for valence (positive or negative), specificity, relation to behaviour and whether the comment suggested a strategy for improvement. Four of the authors independently classified the MSF reports as either 'no reasons for concern' or 'possibly some concern', thereby examining discriminating capacity. Through iterative readings, the authors furthermore tried to identify how many respondents were needed in order to get a reliable impression of a trainee. RESULTS Out of all comments coded for valence (n=1935), 89% were positive and 11% negative. Out of all coded comments (n=4684), 3.8% were suggesting ways to improve. 92% of trainees and supervisors preferred a narrative MSF to a numerical MSF, and 82% of the trainees discovered performance in need of development, but only 53% had made a specific plan for development. Kappa coefficients for inter-rater correlations between four authors were 0.7-1. There was a significant association (p<0.001) between the number of negative comments and the qualitative judgement by the four authors. It was not possible to define a specific number of respondents needed. CONCLUSIONS A purely narrative MSF contributes with educational value and experienced supervisors can discriminate between trainees' performances based on the MSF reports.
Collapse
Affiliation(s)
- Ellen Astrid Holm
- Department of Internal Medicine, Zealand University Hospital Koge, Koge, Denmark
- Institute of Clinical Medicine, University of Copenhagen, Kobenhavns, Denmark
| | | | - Toke Seierøe Barfod
- Department of Internal Medicine, Zealand University Hospital Roskilde, Roskilde, Denmark
| | - Maurice A Lembeck
- Department of Internal Medicine, Nykobing F Sygehus, Nykobing Falster, Denmark
| | - Hanne Pedersen
- Department of Internal Medicine, Glostrup, Rigshospitalet, Kobenhavn, Denmark
| | - Emilie Ramberg
- Department of Internal Medicine, Nykobing F Sygehus, Nykobing Falster, Denmark
| | | | - Jette Led Sorensen
- Juliane Marie Centre for Children, Women and Reproduction Section 4074, Rigshospitalet, Kobenhavn, Denmark
- Children Hospital Copenhagen, Rigshospitalet, Kobenhavn, Denmark
| |
Collapse
|
15
|
Wiese A, Galvin E, O'Farrell J, Cotter J, Bennett D. Doctors' maintenance of professional competence: a qualitative study informed by the theory of planned behaviour. BMC Health Serv Res 2021; 21:419. [PMID: 33941175 PMCID: PMC8094537 DOI: 10.1186/s12913-021-06438-9] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/24/2020] [Accepted: 04/21/2021] [Indexed: 12/15/2022] Open
Abstract
BACKGROUND Medical regulators worldwide have implemented programmes of maintenance of professional competence (MPC) to ensure that doctors, throughout their careers, are up to date and fit to practice. The introduction of MPC required doctors to adopt a range of new behaviours. Despite high enrolment rates on these programmes, it remains uncertain whether doctors engage in the process because they perceive benefits like improvements in their practice and professional development or if they solely meet the requirements to retain medical registration. In this study, we aimed to explore the relationship between doctors' beliefs, intention and behaviour regarding MPC through the lens of the Theory of Planned Behaviour (TPB) to make explicit the factors that drive meaningful engagement with the process. METHODS We conducted a qualitative study using semi-structured interviews. From a pool of 1258 potential participants, we purposively selected doctors from multiple specialities, age groups, and locations across Ireland. We used thematic analysis, and the TPB informed the analytic coding process. RESULTS Forty-one doctors participated in the study. The data analysis revealed doctors' intention and behaviour and the factors that shape their engagement with MPC. We found that attitudes and beliefs about the benefits and impact of MPC mediated the nature of doctors' engagement with the process. Some participants perceived positive changes in practice and other gains from participating in MPC, which facilitated committed engagement with the process. Others believed MPC was unfair, unnecessary, and lacking any benefit, which negatively influenced their intention and behaviour, and that was demonstrated by formalistic engagement with the process. Although participants with positive and negative attitudes shared perceptions about barriers to participation, such perceptions did not over-ride strongly positive beliefs about the benefits of MPC. While the requirements of the regulator strongly motivated doctors to participate in MPC, beliefs about patient expectations appear to have had less impact on intention and behaviour. CONCLUSIONS The findings of this study broaden our understanding of the determinants of doctors' intention and behaviour regarding MPC, which offers a basis for designing targeted interventions. While the barriers to engagement with MPC resonate with previous research findings, our findings challenge critical assumptions about enhancing doctors' engagement with the process. Overall, our results suggest that focused policy initiatives aimed at strengthening the factors that underpin the intention and behaviour related to committed engagement with MPC are warranted.
Collapse
Affiliation(s)
- Anél Wiese
- Medical Education Unit, School of Medicine, University College Cork, Cork, Ireland.
| | - Emer Galvin
- Medical Education Unit, School of Medicine, University College Cork, Cork, Ireland
| | | | | | - Deirdre Bennett
- Medical Education Unit, School of Medicine, University College Cork, Cork, Ireland
| |
Collapse
|
16
|
Carenzo L, Cena T, Carfagna F, Rondi V, Ingrassia PL, Cecconi M, Violato C, Della Corte F, Vaschetto R. Assessing anaesthesiology and intensive care specialty physicians: An Italian language multisource feedback system. PLoS One 2021; 16:e0250404. [PMID: 33891626 PMCID: PMC8064525 DOI: 10.1371/journal.pone.0250404] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/12/2021] [Accepted: 04/07/2021] [Indexed: 11/18/2022] Open
Abstract
BACKGROUND Physician professionalism, including anaesthesiologists and intensive care doctors, should be continuously assessed during training and subsequent clinical practice. Multi-source feedback (MSF) is an assessment system in which healthcare professionals are assessed on several constructs (e.g., communication, professionalism, etc.) by multiple people (medical colleagues, coworkers, patients, self) in their sphere of influence. MSF has gained widespread acceptance for both formative and summative assessment of professionalism for reflecting on how to improve clinical practice. METHODS Instrument development and psychometric analysis (feasibility, reliability, construct validity via exploratory factor analysis) for MSF questionnaires in a postgraduate specialty training in Anaesthesiology and intensive care in Italy. Sixty-four residents at the Università del Piemonte Orientale (Italy) Anesthesiology Residency Program. Main outcomes assessed were: development and psychometric testing of 4 questionnaires: self, medical colleague, coworker and patient assessment. RESULTS Overall 605 medical colleague questionnaires (mean of 9.3 ±1.9) and 543 coworker surveys (mean 8.4 ±1.4) were collected providing high mean ratings for all items (> 4.0 /5.0). The self-assessment item mean score ranged from 3.1 to 4.3. Patient questionnaires (n = 308) were returned from 31 residents (40%; mean 9.9 ± 6.2). Three items had high percentages of "unable to assess" (> 15%) in coworker questionnaires. Factor analyses resulted in a two-factor solution: clinical management with leadership and accountability accounting for at least 75% of the total variance for the medical colleague and coworker's survey with high internal consistency reliability (Cronbach's α > 0.9). Patient's questionnaires had a low return rate, a limited exploratory analysis was performed. CONCLUSIONS We provide a feasible and reliable Italian language MSF instrument with evidence of construct validity for the self, coworkers and medical colleague. Patient feedback was difficult to collect in our setting.
Collapse
Affiliation(s)
- Luca Carenzo
- Department of Anesthesia and Intensive Care Medicine, Humanitas Clinical and Research Center—IRCCS, Rozzano (MI), Italy
- * E-mail:
| | - Tiziana Cena
- Department of Anaesthesia and Intensive Care Medicine, Azienda Ospedaliero-Universitaria “Maggiore della Carità”, Novara, Italy
| | - Fabio Carfagna
- Department of Biomedical Sciences, Humanitas University, Pieve Emanuele–Milan, Italy
| | - Valentina Rondi
- Dipartimento di Medicina Traslazionale, Università del Piemonte Orientale, Novara, Italy
| | - Pier Luigi Ingrassia
- Centro di Simulazione, Centro Professionale Sociosanitario, Lugano, Switzerland
- Centro Interdipartimentale di Didattica Innovativa e di Simulazione in Medicina e Professioni Sanitarie, SIMNOVA, Università del Piemonte Orientale, Novara, Italy
| | - Maurizio Cecconi
- Department of Anesthesia and Intensive Care Medicine, Humanitas Clinical and Research Center—IRCCS, Rozzano (MI), Italy
- Department of Biomedical Sciences, Humanitas University, Pieve Emanuele–Milan, Italy
| | - Claudio Violato
- Departments of Medicine and Medical Education, University of Minnesota Medical School, Minneapolis, MN, United States of America
| | - Francesco Della Corte
- Department of Anaesthesia and Intensive Care Medicine, Azienda Ospedaliero-Universitaria “Maggiore della Carità”, Novara, Italy
- Dipartimento di Medicina Traslazionale, Università del Piemonte Orientale, Novara, Italy
| | - Rosanna Vaschetto
- Department of Anaesthesia and Intensive Care Medicine, Azienda Ospedaliero-Universitaria “Maggiore della Carità”, Novara, Italy
- Dipartimento di Medicina Traslazionale, Università del Piemonte Orientale, Novara, Italy
| |
Collapse
|
17
|
Dai CM, Bertram K, Chahine S. Feedback Credibility in Healthcare Education: a Systematic Review and Synthesis. MEDICAL SCIENCE EDUCATOR 2021; 31:923-933. [PMID: 34457934 PMCID: PMC8368112 DOI: 10.1007/s40670-020-01167-w] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 11/19/2020] [Indexed: 05/21/2023]
Abstract
PURPOSE The purpose of this study was to systematically review and synthesize factors that influence learners' perceptions of credibility when feedback is provided by an authority figure in a healthcare environment. METHODS This study reviewed literature from medicine, psychology, and education using systematic review and qualitative synthesis methods. In a multi-step process, major electronic bibliographic databases were searched for relevant studies until October 2020. RESULTS The search identified 9216 articles. A total of 134 abstracts underwent full-text review. Of these, 22 articles met inclusion criteria. The studies were heterogenous and the majority utilized a qualitative design with interviews and focus groups. A few studies employed mixed methodology (n = 2) and two studies used a quantitative design. Four main themes were identified: feedback characteristics, context of feedback, source credibility, and recipient characteristics. CONCLUSION As programs implement major educational change initiatives to create more formative assessment practices, feedback will become even more crucial. The four main themes identified are important factors that contribute to the perception of feedback credibility. While the factors are described independently, they may be viewed as interrelated and the association between these factors and feedback may be driven more by learning culture than each characteristic. SUPPLEMENTARY INFORMATION The online version contains supplementary material available at 10.1007/s40670-020-01167-w.
Collapse
Affiliation(s)
- Cecilia M. Dai
- Schulich School of Medicine & Dentistry, Western University, London, Ontario Canada
- Health Sciences Addition, Western University, Room H110B, London, Ontario N6A 5C1 Canada
| | - Kaitlyn Bertram
- Royal Victoria Regional Health Centre (RVH), University of Toronto Department of Family and Community Medicine, Toronto, Ontario Canada
| | - Saad Chahine
- Centre for Education Research & Innovation, Schulich School of Medicine & Dentistry, Western University, London, Ontario Canada
| |
Collapse
|
18
|
Yin AL, Gheissari P, Lin IW, Sobolev M, Pollak JP, Cole C, Estrin D. Role of Technology in Self-Assessment and Feedback Among Hospitalist Physicians: Semistructured Interviews and Thematic Analysis. J Med Internet Res 2020; 22:e23299. [PMID: 33141098 PMCID: PMC7671832 DOI: 10.2196/23299] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2020] [Revised: 10/04/2020] [Accepted: 10/09/2020] [Indexed: 12/26/2022] Open
Abstract
Background Lifelong learning is embedded in the culture of medicine, but there are limited tools currently available for many clinicians, including hospitalists, to help improve their own practice. Although there are requirements for continuing medical education, resources for learning new clinical guidelines, and developing fields aimed at facilitating peer-to-peer feedback, there is a gap in the availability of tools that enable clinicians to learn based on their own patients and clinical decisions. Objective The aim of this study was to explore the technologies or modifications to existing systems that could be used to benefit hospitalist physicians in pursuing self-assessment and improvement by understanding physicians’ current practices and their reactions to proposed possibilities. Methods Semistructured interviews were conducted in two separate stages with analysis performed after each stage. In the first stage, interviews (N=12) were conducted to understand the ways in which hospitalist physicians are currently gathering feedback and assessing their practice. A thematic analysis of these interviews informed the prototype used to elicit responses in the second stage. Results Clinicians actively look for feedback that they can apply to their practice, with the majority of the feedback obtained through self-assessment. The following three themes surrounding this aspect were identified in the first round of semistructured interviews: collaboration, self-reliance, and uncertainty, each with three related subthemes. Using a wireframe, the second round of interviews led to identifying the features that are currently challenging to use or could be made available with technology. Conclusions Based on each theme and subtheme, we provide targeted recommendations for use by relevant stakeholders such as institutions, clinicians, and technologists. Most hospitalist self-assessments occur on a rolling basis, specifically using data in electronic medical records as their primary source. Specific objective data points or subjective patient relationships lead clinicians to review their patient cases and to assess their own performance. However, current systems are not built for these analyses or for clinicians to perform self-assessment, making this a burdensome and incomplete process. Building a platform that focuses on providing and curating the information used for self-assessment could help physicians make more accurately informed changes to their own clinical practice and decision-making.
Collapse
Affiliation(s)
- Andrew Lukas Yin
- Medical College, Weill Cornell Medicine, New York, NY, United States.,Cornell Tech, New York, NY, United States
| | | | | | - Michael Sobolev
- Cornell Tech, New York, NY, United States.,Feinstein Institutes for Medical Research, Northwell Health, Manhasset, NY, United States
| | - John P Pollak
- Cornell Tech, New York, NY, United States.,Department of Medicine, Weill Cornell Medicine, New York, NY, United States
| | - Curtis Cole
- Department of Medicine, Weill Cornell Medicine, New York, NY, United States.,Department of Population Health Sciences, Weill Cornell Medicine, New York, NY, United States
| | - Deborah Estrin
- Cornell Tech, New York, NY, United States.,Department of Population Health Sciences, Weill Cornell Medicine, New York, NY, United States
| |
Collapse
|
19
|
Hennel EK, Subotic U, Berendonk C, Stricker D, Harendza S, Huwendiek S. A german-language competency-based multisource feedback instrument for residents: development and validity evidence. BMC MEDICAL EDUCATION 2020; 20:357. [PMID: 33046060 PMCID: PMC7552497 DOI: 10.1186/s12909-020-02259-2] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/22/2019] [Accepted: 09/28/2020] [Indexed: 06/11/2023]
Abstract
BACKGROUND In medical settings, multisource feedback (MSF) is a recognised method of formative assessment. It collects feedback on a doctor's performance from several perspectives in the form of questionnaires. Yet, no validated MSF questionnaire has been publicly available in German. Thus, we aimed to develop a German MSF questionnaire based on the CanMEDS roles and to investigate the evidence of its validity. METHODS We developed a competency-based MSF questionnaire in German, informed by the literature and expert input. Four sources of validity evidence were investigated: (i) Content was examined based on MSF literature, blueprints of competency, and expert-team discussions. (ii) The response process was supported by analysis of a think-aloud study, narrative comments, "unable to comment" ratings and evaluation data. (iii) The internal structure was assessed by exploratory factor analysis, and inter-rater reliability by generalisability analysis. Data were collected during two runs of MSF, in which 47 residents were evaluated once (first run) or several times (second and third run) on 81 occasions of MSF. (iv) To investigate consequences, we analysed the residents' learning goals and the progress as reported via MSF. RESULTS Our resulting MSF questionnaire (MSF-RG) consists of 15 items and one global rating, which are each rated on a scale and accompanied by a field for narrative comments and cover a construct of a physician's competence. Additionally, there are five open questions for further suggestions. Investigation of validity evidence revealed that: (i) The expert group agreed that the content comprehensively addresses clinical competence; (ii) The response processes indicated that the questions are understood as intended and supported the acceptance and usability; (iii) For the second run, factor analysis showed a one-factor solution, a Cronbach's alpha of 0.951 and an inter-rater reliability of 0.797 with 12 raters; (iv) There are indications that residents benefitted, considering their individual learning goals and based on their ratings reported via MSF itself. CONCLUSIONS To support residency training with multisource feedback, we developed a German MSF questionnaire (MSF-RG), which is supported by four sources of validity evidence. This MSF questionnaire may be useful to implement MSF in residency training in German-speaking regions.
Collapse
Affiliation(s)
- Eva K. Hennel
- Department for Assessment and Evaluation (AAE), Institute for Medical Education, University of Bern, Mittelstrasse 43, 3012 Bern, Switzerland
| | - Ulrike Subotic
- University Children’s Hospital Basel, Spitalstrasse 33, 4056 Basel, Switzerland
| | - Christoph Berendonk
- Department for Assessment and Evaluation (AAE), Institute for Medical Education, University of Bern, Mittelstrasse 43, 3012 Bern, Switzerland
| | - Daniel Stricker
- Department for Assessment and Evaluation (AAE), Institute for Medical Education, University of Bern, Mittelstrasse 43, 3012 Bern, Switzerland
| | - Sigrid Harendza
- Department of Internal Medicine, University Medical Centre Hamburg-Eppendorf, Martinistr. 52, 20246 Hamburg, Germany
| | - Sören Huwendiek
- Department for Assessment and Evaluation (AAE), Institute for Medical Education, University of Bern, Mittelstrasse 43, 3012 Bern, Switzerland
| |
Collapse
|
20
|
Tzortziou Brown V, McCartney M, Heneghan C. Appraisal and revalidation for UK doctors—time to assess the evidence. BRITISH MEDICAL JOURNAL 2020. [DOI: 10.1136/bmj.m3415] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/03/2022]
|
21
|
Lai MMY, Roberts N, Mohebbi M, Martin J. A randomised controlled trial of feedback to improve patient satisfaction and consultation skills in medical students. BMC MEDICAL EDUCATION 2020; 20:277. [PMID: 32819352 PMCID: PMC7439652 DOI: 10.1186/s12909-020-02171-9] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/12/2019] [Accepted: 07/22/2020] [Indexed: 05/30/2023]
Abstract
BACKGROUND The use of feedback has been integral to medical student learning, but rigorous evidence to evaluate its education effect is limited, especially in the role of patient feedback in clinical teaching and practice improvement. The aim of the Patient Teaching Associate (PTA) Feedback Study was to evaluate whether additional written consumer feedback on patient satisfaction improved consultation skills among medical students and whether multisource feedback (MSF) improved student performance. METHODS In this single site, double-blinded randomised controlled trial, 71 eligible medical students from two universities in their first clinical year were allocated to intervention or control and followed up for one semester. They participated in five simulated student-led consultations in a teaching clinic with patient volunteers living with chronic illness. Students in the intervention group received additional written feedback on patient satisfaction combined with guided self-reflection. The control group received usual immediate formative multisource feedback from tutors, patients and peers. Student characteristics, baseline patient-rated satisfaction scores and tutor-rated consultation skills were measured. RESULTS Follow-up assessments were complete in 70 students attending the MSF program. At the final consultation episodes, both groups improved patient-rated rapport (P = 0.002), tutor-rated patient-centeredness and tutor-rated overall consultation skills (P = 0.01). The intervention group showed significantly better tutor-rated patient-centeredness (P = 0.003) comparing with the control group. Distress relief, communication comfort, rapport reported by patients and tutor-rated clinical skills did not differ significantly between the two groups. CONCLUSIONS The innovative multisource feedback program effectively improved consultation skills in medical students. Structured written consumer feedback combined with guided student reflection further improved patient-centred practice and effectively enhanced the benefit of an MSF model. This strategy might provide a valuable adjunct to communication skills education for medical students. TRIAL REGISTRATION Australian New Zealand Clinical Trials Registry Number ACTRN12613001055796 .
Collapse
Affiliation(s)
- Michelle M Y Lai
- Medical Student Programs, Eastern Health Clinical School, Monash University and Deakin University, Level 2, Arnold Street, Box Hill, VIC, 3128, Australia.
- Curtin University Medical School, Perth, Australia.
| | - Noel Roberts
- Medical Student Programs, Eastern Health Clinical School, Monash University and Deakin University, Level 2, Arnold Street, Box Hill, VIC, 3128, Australia
| | | | - Jenepher Martin
- Medical Student Programs, Eastern Health Clinical School, Monash University and Deakin University, Level 2, Arnold Street, Box Hill, VIC, 3128, Australia
| |
Collapse
|
22
|
Bion J, Brookes O, Brown C, Tarrant C, Archer J, Buckley D, Buckley LM, Clement I, Evison F, Smith FG, Gibbins C, Hayton EJ, Jones J, Lilford R, Mullhi R, Packer G, Perkins GD, Shelton J, Snelson C, Sullivan P, Vlaev I, Wolstenholme D, Wright S. A framework and toolkit of interventions to enhance reflective learning among health-care professionals: the PEARL mixed-methods study. HEALTH SERVICES AND DELIVERY RESEARCH 2020. [DOI: 10.3310/hsdr08320] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/21/2022] Open
Abstract
Background
Although most health care is high quality, many patients and members of staff can recall episodes of a lack of empathy, respect or effective communication from health-care staff. In extreme form, this contributes to high-profile organisational failures. Reflective learning is a universally promoted technique for stimulating insight, constructive self-appraisal and empathy; however, its efficacy tends to be assumed rather than proven. The Patient Experience And Reflective Learning (PEARL) project has used patient and staff experience to co-design a novel reflective learning framework that is based on theories of behaviour and learning.
Objective
To create a toolkit to help health-care staff obtain meaningful feedback to stimulate effective reflective learning that will promote optimal patient-, family- and colleague-focused behaviours.
Design
A 3-year developmental mixed-methods study with four interlinked workstreams and 12 facilitated co-design meetings. The Capability, Opportunity, Motivation – Behaviour framework was used to describe factors influencing the behaviour of reflection.
Setting
This took place at five acute medical units and three intensive care units in three urban acute hospital trusts in England.
Participants
Patients and relatives, medical and nursing staff, managers and researchers took part.
Data sources
Two anonymous surveys, one for patients and one for staff, were developed from existing UK-validated instruments, administered locally and analysed centrally. Ethnographers undertook interviews and observed clinical care and reflective learning activities in the workplace, as well as in the co-design meetings, and fed back their observations in plenary workshops.
Main outcome measures
Preliminary instruments were rated by participants for effectiveness and feasibility to derive a final set of tools. These are presented in an attractively designed toolbox with multiple sections, including the theoretical background of reflection, mini guides for obtaining meaningful feedback and for reflecting effectively, guides for reflecting ‘in-action’ during daily activities, and a set of resources.
Results
Local project teams (physicians, nurses, patients, relatives and managers) chaired by a non-executive director found the quarterly reports of feedback from the patient and staff surveys insightful and impactful. Patient satisfaction with care was higher for intensive care units than for acute medical units, which reflects contextual differences, but in both settings quality of communication was the main driver of satisfaction. Ethnographers identified many additional forms of experiential feedback. Those that generated an emotional response were particularly effective as a stimulus for reflection. These sources of data were used to supplement individual participant experiences in the nine local co-design meetings and four workshops to identify barriers to and facilitators of effective reflection, focusing on capability, opportunity and motivation. A logic model was developed combining the Capability, Opportunity, Motivation – Behaviour framework for reflection and theories of learning to link patient and staff experience to changes in downstream behaviours. Participants proposed practical tools and activities to enhance reflection ‘in-action’ and ‘on-action’. These tools were developed iteratively by the local and central project teams.
Limitations
Paper-based surveys were burdensome to administer and analyse.
Conclusions
Patients and health-care staff collaborated to produce a novel reflective learning toolkit.
Future work
The toolkit requires evaluating in a cluster randomised controlled trial.
Funding
This project was funded by the National Institute for Health Research (NIHR) Health Services and Delivery Research programme and will be published in full in Health Services and Delivery Research; Vol. 8, No. 32. See the NIHR Journals Library website for further project information.
Collapse
Affiliation(s)
- Julian Bion
- Department of Anaesthesia & Intensive Care Medicine, University of Birmingham, Birmingham, UK
| | - Olivia Brookes
- University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
| | - Celia Brown
- Population Evidence and Technologies, University of Warwick, Coventry, UK
| | - Carolyn Tarrant
- Social Science Applied to Healthcare Improvement Research (SAPPHIRE) Group, Department of Health Sciences, University of Leicester, Leicester, UK
| | - Julian Archer
- Royal Australasian College of Surgeons, Melbourne, VIC, Australia
| | - Duncan Buckley
- Patient and Public Involvement Representative, Birmingham, UK
| | | | - Ian Clement
- Critical Care, Newcastle upon Tyne Hospitals NHS Foundation Trust, Newcastle upon Tyne, UK
| | - Felicity Evison
- Informatics Department, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
| | - Fang Gao Smith
- Institute of Inflammation and Ageing, University of Birmingham, Birmingham, UK
| | - Chris Gibbins
- Acute Medicine, Newcastle upon Tyne Hospitals NHS Foundation Trust, Newcastle upon Tyne, UK
| | - Emma-Jo Hayton
- Acute Medicine, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
| | - Jennifer Jones
- Social Science Applied to Healthcare Improvement Research (SAPPHIRE) Group, Department of Health Sciences, University of Leicester, Leicester, UK
| | - Richard Lilford
- Warwick Centre for Applied Health Research and Delivery, University of Warwick, Coventry, UK
| | - Randeep Mullhi
- Critical Care, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
| | - Greg Packer
- Critical Care, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
| | - Gavin D Perkins
- Critical Care Medicine, Warwick Medical School, Warwick Clinical Trials Unit, University of Warwick, Coventry, UK
| | - Jonathan Shelton
- Critical Care, Newcastle upon Tyne Hospitals NHS Foundation Trust, Newcastle upon Tyne, UK
| | - Catherine Snelson
- Acute Medicine, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
- Critical Care, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
| | - Paul Sullivan
- Acute Medicine, Chelsea and Westminster Hospital NHS Foundation Trust, London, UK
| | - Ivo Vlaev
- Behavioural Science Group, University of Warwick, Coventry, UK
| | - Daniel Wolstenholme
- National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care Yorkshire and Humber, Sheffield, UK
| | - Stephen Wright
- Critical Care, Newcastle upon Tyne Hospitals NHS Foundation Trust, Newcastle upon Tyne, UK
| | | |
Collapse
|
23
|
Ashworth N, Kain NA, Jess E, Mazurek K. Survey of physician attitudes to using multisource feedback for competence assessment in Alberta. BMJ Open 2020; 10:e037610. [PMID: 32690531 PMCID: PMC7371216 DOI: 10.1136/bmjopen-2020-037610] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/02/2022] Open
Abstract
BACKGROUND The use of multisource feedback (MSF) for assessing physician performance is widespread and rapidly growing. Findings from early very small research studies using highly selected participants suggest high levels of satisfaction and support. However, after nearly two decades of experience using MSF to evaluate all physicians in Alberta, we are sceptical of this. OBJECTIVES To determine physicians' actual opinions of MSF using the entire physician population of Alberta, Canada DESIGN: Online survey. SETTING Alberta, Canada. PARTICIPANTS All physicians with a full licence to practice in Alberta in 2015. INTERVENTIONS All participants were asked to grade how well they thought MSF was at assessing various aspects of physician performance using a 10-point Likert-type scale. There was also a text response field for written comments. OUTCOMES Mean responses to quantitative questions. Qualitative content and thematic analysis of open-ended text responses.We analysed the data using SPSS V.23 and NVivo V.11 and built a multivariate model highlighting the predictors of high and low opinions of MSF. RESULTS Survey response rate was high for physicians with 2215 responses (25%). The mean rating for how successful MSF was at assessing a variety of dimensions, varied from a low of 5.03/10 for medical knowledge to a high of 6.38/10 for professionalism and communication. Canadian-trained MDs rated MSF significantly lower on every dimension by approximately 20% compared with non-Canadian-trained MDs. CONCLUSIONS Alberta physicians have much lower opinions about the ability of MSF to measure any dimension of their performance than what has been suggested in the literature. Canadian-trained MDs have a particularly low opinion of MSF for reasons that remain unclear. The results of this survey offer a serious challenge to the effectiveness of a programme that is designed to promote self-reflection and performance improvement.
Collapse
Affiliation(s)
- Nigel Ashworth
- Department of Medicine, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Alberta, Canada
- Research and evaluation unit, College of Physicians and Surgeons of Alberta, Edmonton, Alberta, Canada
| | - Nicole Allison Kain
- Department of Medicine, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Alberta, Canada
- Research and evaluation unit, College of Physicians and Surgeons of Alberta, Edmonton, Alberta, Canada
| | - Ed Jess
- Research and evaluation unit, College of Physicians and Surgeons of Alberta, Edmonton, Alberta, Canada
| | - Karen Mazurek
- Research and evaluation unit, College of Physicians and Surgeons of Alberta, Edmonton, Alberta, Canada
| |
Collapse
|
24
|
Primhak R. Assessment methods in respiratory medicine training in Europe: current status and future needs. Breathe (Sheff) 2020; 16:190314. [PMID: 32494302 PMCID: PMC7249786 DOI: 10.1183/20734735.0314-2019] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022] Open
Abstract
There is a perceived need for harmonisation of training standards in medicine across Europe. Assessment methodology is a fundamental part of this harmonisation. Assessment may be: summative, measuring competency at the end of training, usually in a pass/fail mode; or formative, assessing strengths and weaknesses during training with a view to optimising performance. A survey of experienced respiratory medicine trainers in Europe was undertaken to discover the formative and summative assessment methods being used, and to explore the use of workplace-based assessments (WBAs). Structured interviews were sought with experienced trainers in adult and paediatric respiratory medicine. 35 trainers from 22 (71%) out of 31 countries were interviewed. The number of types of summative assessments required at the end of training varied from 0 to 4 including clinical (25%), written (44%) and oral (47%) examinations. Four respondents required a research thesis as a proof of clinical competence. WBA was not commonly used. Only 14% of respondents reported using a formal case-based discussion, 20% used a form of multisource feedback and 25% described some form of formal assessment of procedural skills. However, 77% of all respondents expressed a wish to have access to case-based discussion and 72% wanted to have a facility for multisource feedback. The majority also wanted training in the use of these tools. Almost half of the respondents had received no formal training in educational supervision and 80% of all respondents expressed a wish to receive such training. The findings suggest that there is no adequate process of ensuring uniform standards for specialist accreditation in Europe, and demonstrate a need and desire among trainers for more WBA tools and training in their use to be made available.
Collapse
Affiliation(s)
- Robert Primhak
- Education Council, European Respiratory Society, Lausanne, Switzerland
| |
Collapse
|
25
|
Atsma F, Elwyn G, Westert G. Understanding unwarranted variation in clinical practice: a focus on network effects, reflective medicine and learning health systems. Int J Qual Health Care 2020; 32:271-274. [PMID: 32319525 PMCID: PMC7270826 DOI: 10.1093/intqhc/mzaa023] [Citation(s) in RCA: 41] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2019] [Revised: 02/10/2020] [Accepted: 02/26/2020] [Indexed: 12/16/2022] Open
Abstract
In the past decades, extensive research has been performed on the phenomenon of unwarranted clinical variation in clinical practice. Many studies have been performed on signaling, describing and visualizing clinical variation. We argue that it is time for next steps in practice variation research. In addition to describing and signaling variation patterns, we argue that a better understanding of causes of variation should be gained. Moreover, target points for improving and decreasing clinical variation should be created. Key elements in this new focus should be research on the complex interaction of networks, reflective medicine, patient beliefs and objective criteria for treatment choices. By combining these different concepts, alternative research objectives and new targets for improving and reducing unwarranted variation may be defined. In this perspective, we reflect on these concepts and propose target points for future research.
Collapse
Affiliation(s)
- Femke Atsma
- Scientific Center for Quality of Healthcare, Radboud University Medical Center, Radboud Institute for Health Sciences, Geert Grooteplein Noord 21, P.O. Box 9101, 6500 HB Nijmegen, The Netherlands
| | - Glyn Elwyn
- Scientific Center for Quality of Healthcare, Radboud University Medical Center, Radboud Institute for Health Sciences, Geert Grooteplein Noord 21, P.O. Box 9101, 6500 HB Nijmegen, The Netherlands
- The Dartmouth Institute for Health Policy and Clinical Practice, Dartmouth College, 1 Medical Center Drive, Lebanon, NH, 03756, USA
| | - Gert Westert
- Scientific Center for Quality of Healthcare, Radboud University Medical Center, Radboud Institute for Health Sciences, Geert Grooteplein Noord 21, P.O. Box 9101, 6500 HB Nijmegen, The Netherlands
| |
Collapse
|
26
|
Ratelle JT, Halvorsen AJ, Mandrekar J, Sawatsky AP, Reed DA, Beckman TJ. Internal Medicine Resident Professionalism Assessments: Exploring the Association With Patients' Overall Satisfaction With Their Hospital Stay. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2020; 95:902-910. [PMID: 31809293 DOI: 10.1097/acm.0000000000003114] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
PURPOSE Successful training of internal medicine (IM) residents requires accurate assessments. Patients could assess IM residents in a hospital setting, but medical educators must understand how contextual factors may affect assessments. The objective was to investigate relationships between patient, resident, and hospital-encounter characteristics and the results of patient assessments of IM resident professionalism. METHOD The authors performed a prospective cohort study of postgraduate year 1 (PGY-1) IM residents and their patients at 4 general medicine inpatient teaching services at Mayo Clinic Hospital-Rochester, Saint Marys Campus in Rochester, Minnesota, from July 1, 2015, through June 30, 2016. Patient assessments of resident professionalism were adapted from validated instruments. Multivariable modeling with generalized estimating equations was used to determine associations between patient assessment scores and characteristics of residents, residents' clinical performance and evaluations (including professionalism assessments in other settings), patients, and hospital encounters and to account for repeated assessments of residents. RESULTS A total of 409 patients assessed 72 PGY-1 residents (mean [SD], 5.7 [3.0] patient assessments per resident). In the multivariable model, only the highest rating out of 5 levels for overall satisfaction with hospital stay was significantly associated with patient assessment scores of resident professionalism (β [SE], 0.80 [0.08]; P < .001). Hospitalized patients' assessment scores of resident professionalism were not significantly correlated with assessment scores of resident professionalism in other clinical settings. CONCLUSIONS Hospitalized patients' assessment scores of in-hospital resident professionalism were strongly correlated with overall patient satisfaction with hospital stay but were not correlated with resident professionalism in other settings. The limitations of patient evaluations should be considered before incorporating these evaluations into programs of assessment.
Collapse
Affiliation(s)
- John T Ratelle
- J.T. Ratelle is a consultant, Division of Hospital Internal Medicine, Mayo Clinic, and assistant professor of medicine, Mayo Clinic College of Medicine and Science, Rochester, Minnesota. A.J. Halvorsen is a statistician, Internal Medicine Residency Office, Mayo Clinic, and assistant professor of medicine, Mayo Clinic College of Medicine and Science, Rochester, Minnesota. J. Mandrekar is a consultant, Division of Biomedical Statistics and Informatics, Mayo Clinic, and professor of biostatistics and neurology, Mayo Clinic College of Medicine and Science, Rochester, Minnesota. A.P. Sawatsky is a consultant, Division of General Internal Medicine, Mayo Clinic, and assistant professor of medicine, Mayo Clinic College of Medicine and Science, Rochester, Minnesota. D.A. Reed is a consultant, Division of Community Internal Medicine, Mayo Clinic, and associate professor of medical education and professor of medicine, Mayo Clinic College of Medicine and Science, Rochester, Minnesota. T.J. Beckman is a consultant, Division of General Internal Medicine, Mayo Clinic, and professor of medical education and medicine, Mayo Clinic College of Medicine and Science, Rochester, Minnesota
| | | | | | | | | | | |
Collapse
|
27
|
Ledford R, Burger A, LaRochelle J, Klocksieben F, DeWaay D, O’Brien KE. Exploring Perspectives from Internal Medicine Clerkship Directors in the USA on Effective Narrative Evaluation: Results from the CDIM National Survey. MEDICAL SCIENCE EDUCATOR 2020; 30:155-161. [PMID: 34457654 PMCID: PMC8368638 DOI: 10.1007/s40670-019-00825-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
PURPOSE Clinical performance evaluations play a critical role in determining medical school clerkship grades. This study aimed to provide clarification from clerkship directors in internal medicine on what constitutes an effective and informative narrative description of student performance. METHODS In September 2016, the Clerkship Directors in Internal Medicine (CDIM) electronically administered its annual, voluntary, and confidential cross-sectional survey of its US membership. One section of the survey asked six questions regarding the helpful components of an effective narrative evaluation. Respondents were asked to rate the effectiveness of elements contained within narrative evaluations of students. RESULTS Ninety-five CDIM members responded to the survey with an overall response rate of 74.2%. Descriptions of skills and behaviors were felt to be the most important, followed by a description of the overall synthetic or global assessment level of the student. Descriptions of personality and attitude were the next highest rated feature followed by adjectives describing performance. Length was felt to be the least important component. In free-text comments, several respondents indicated that direct observation of performance and specific examples of skills and behaviors are also desirable. CONCLUSIONS Narrative evaluations of students that explicitly comment on skills, behaviors, and an overarching performance level of the learner are strongly preferred by clerkship directors. Direct observation of clinical performance and giving specific examples of such behaviors give evaluations even more importance. Faculty development on evaluation and assessment should include instruction on these narrative assessment characteristics.
Collapse
Affiliation(s)
- Robert Ledford
- Department of Internal Medicine, Division of Hospital Medicine, University of South Florida Morsani College of Medicine, 12901 Bruce B. Downs Boulevard, MDC 80, Tampa, FL 33612 USA
| | - Alfred Burger
- Icahn School of Medicine at Mount Sinai, New York, NY USA
| | - Jeff LaRochelle
- Department of Medical Education, University of Central Florida College of Medicine, Orlando, FL USA
| | - Farina Klocksieben
- Department of Internal Medicine, Division of Hospital Medicine, University of South Florida Morsani College of Medicine, 12901 Bruce B. Downs Boulevard, MDC 80, Tampa, FL 33612 USA
| | - Deborah DeWaay
- Department of Internal Medicine, Division of Hospital Medicine, University of South Florida Morsani College of Medicine, 12901 Bruce B. Downs Boulevard, MDC 80, Tampa, FL 33612 USA
| | - Kevin E. O’Brien
- Department of Internal Medicine, Division of Hospital Medicine, University of South Florida Morsani College of Medicine, 12901 Bruce B. Downs Boulevard, MDC 80, Tampa, FL 33612 USA
| |
Collapse
|
28
|
Lockyer J, DiMillo S, Campbell C. An Examination of Self-Reported Assessment Activities Documented by Specialist Physicians for Maintenance of Certification. THE JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS 2020; 40:19-26. [PMID: 32149945 DOI: 10.1097/ceh.0000000000000283] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
INTRODUCTION Specialists in a Maintenance of Certification program are required to participate in assessment activities, such as chart audit, simulation, knowledge assessment, and multisource feedback. This study examined data from five different specialties to identify variation in participation in assessment activities, examine differences in the learning stimulated by assessment, assess the frequency and type of planned changes, and assess the association between learning, discussion, and planned changes. METHODS E-portfolio data were categorized and analyzed descriptively. Chi-squared tests examined associations. RESULTS A total of 2854 anatomical pathologists, cardiologists, gastroenterologists, ophthalmologists, and orthopedic surgeons provided data about 6063 assessment activities. Although there were differences in the role that learning played by discipline and assessment type, the most common activities documented across all specialties were self-assessment programs (n = 2122), feedback on teaching (n = 1078), personal practice assessments which the physician did themselves (n = 751), annual reviews (n = 682), and reviews by third parties (n = 661). Learning occurred for 93% of the activities and was associated with change. For 2126 activities, there were planned changes. Activities in which there was a discussion with a peer or supervisor were more likely to result in a change. CONCLUSIONS AND DISCUSSION Although specialists engaged in many types of assessment activities to meet the Maintenance of Certification program requirements, there was variability in how assessment stimulated learning and planned changes. It seems that peer discussion may be an important component in fostering practice change and forming plans for improvement which bears further study.
Collapse
Affiliation(s)
- Jocelyn Lockyer
- Dr. Lockyer: Professor, Department of Community Health Sciences, Cumming School of Medicine, Calgary, Canada. Ms. DiMillo: Senior Data and Research Analyst, Health Policy and Advocacy, Royal College of Physicians and Surgeons of Canada, Ottawa, Canada. Dr. Campbell: Principal Senior Advisor, Competency-based CPD and interim Director, Continuing Professional Development, Office of Specialty Education, Royal College of Physicians and Surgeons of Canada, and Associate Professor, Department of Medicine, University of Ottawa, Ottawa, Canada
| | | | | |
Collapse
|
29
|
Sehlbach C, Govaerts MJB, Mitchell S, Teunissen TGJ, Smeenk FWJM, Driessen EW, Rohde GGU. Perceptions of people with respiratory problems on physician performance evaluation-A qualitative study. Health Expect 2019; 23:247-255. [PMID: 31747110 PMCID: PMC6978864 DOI: 10.1111/hex.12999] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2019] [Revised: 10/17/2019] [Accepted: 10/18/2019] [Indexed: 01/26/2023] Open
Abstract
BACKGROUND Despite increasing calls for patient and public involvement in health-care quality improvement, the question of how patient evaluations can contribute to physician learning and performance assessment has received scant attention. OBJECTIVE The objective of this study was to explore, amid calls for patient involvement in quality assurance, patients' perspectives on their role in the evaluation of physician performance and to support physicians' learning and decision making on professional competence. DESIGN A qualitative study based on semi-structured interviews. SETTING AND PARTICIPANTS The study took place in a secondary care setting in the Netherlands. The authors selected 25 patients from two Dutch hospitals and through the Dutch Lung Foundation, using purposive sampling. METHODS Data were analysed according to the principles of template analysis, based on an a priori coding framework developed from the literature about patient empowerment, feedback and performance assessment. RESULTS The analysis unearthed three predominant patient perspectives: the proactive perspective, the restrained perspective and the outsider perspective. These perspectives differed in terms of perceived power dynamics within the doctor-patient relationship, patients' perceived ability, and willingness to provide feedback and evaluate their physician's performance. Patients' perspectives thus affected the role patients envisaged for themselves in evaluating physician performance. DISCUSSION AND CONCLUSION Although not all patients are equally suitable or willing to be involved, patients can play a role in evaluating physician performance and continuing training through formative approaches. To involve patients successfully, it is imperative to distinguish between different patient perspectives and empower patients by ensuring a safe environment for feedback.
Collapse
Affiliation(s)
- Carolin Sehlbach
- Department of Educational Development and Research, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, The Netherlands.,School of Health Professions Education, Maastricht University, Maastricht, The Netherlands
| | - Marjan J B Govaerts
- Department of Educational Development and Research, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, The Netherlands
| | | | - Truus G J Teunissen
- Patient Contributor, and Researcher at the Department of Medical Humanities, Amsterdam Public Health research institute (APH), Amsterdam UMC Free University Medical Centre, Amsterdam, The Netherlands
| | - Frank W J M Smeenk
- School of Health Professions Education, Maastricht University, Maastricht, The Netherlands.,Respiratory Medicine, Catharina Hospital, Eindhoven, The Netherlands
| | - Erik W Driessen
- Department of Educational Development and Research, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, The Netherlands
| | - Gernot G U Rohde
- Department of Respiratory Medicine, University Hospital, Goethe University, Frankfurt, Germany
| |
Collapse
|
30
|
Beaulieu AM, Kim BS, Topor DR, Dickey CC. Seeing is believing: an exploration of what residents value when they receive feedback. ACADEMIC PSYCHIATRY : THE JOURNAL OF THE AMERICAN ASSOCIATION OF DIRECTORS OF PSYCHIATRIC RESIDENCY TRAINING AND THE ASSOCIATION FOR ACADEMIC PSYCHIATRY 2019; 43:507-511. [PMID: 31161575 DOI: 10.1007/s40596-019-01071-5] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/12/2019] [Revised: 05/03/2019] [Accepted: 05/16/2019] [Indexed: 06/09/2023]
Abstract
OBJECTIVE The primary purpose of this paper is to report on psychiatry residents' perceptions of what is important when receiving feedback from evaluators. METHODS In January 2018, as part of the Harvard South Shore Psychiatry Residency Training Program's (HSS) ongoing local quality improvement efforts to enhance the delivery and effectiveness of feedback that residents receive from faculty during training, the authors disseminated a survey to psychiatry residents (n = 31) at HSS. Residents rated the level of importance of 17 statements pertaining to the way feedback is delivered. Two open-ended prompts also allowed respondents to share examples of growth-oriented and unhelpful feedback they have received during residency. RESULTS Twenty-seven residents responded (87% response rate). Eighty-one percent rated "the evaluator models the same behavior they're encouraging" as "extremely important" when receiving feedback. Many residents also rated the following survey items as "extremely important": "confidence in the evaluator's clinical and interpersonal skills" (63.0%), "amount of time the evaluator observed the resident" (51.9%), "there is a way to fix a performance deficit" (51.8%), and "specific feedback based off the resident's work" (48.1%). Conversely, only 11.1% of residents rated the feedback sandwich as "extremely important." CONCLUSIONS Despite a small sample size, this project demonstrated that, when receiving feedback, the majority of psychiatry residents strongly value when evaluators model the targeted behavior. The feedback sandwich was least important to residents. This project underscores the importance of evaluators serving as role models in the context of feedback, and findings can be used in faculty development activities focused on feedback delivery best practices.
Collapse
Affiliation(s)
- Ashley M Beaulieu
- VA Boston Healthcare System, Harvard Medical School, Boston, MA, USA.
| | - Bo S Kim
- VA Boston Healthcare System, Harvard Medical School, Boston, MA, USA
| | - David R Topor
- VA Boston Healthcare System, Harvard Medical School, Boston, MA, USA
| | - Chandlee C Dickey
- Schulich School of Medicine & Dentistry, University of Western Ontario, London, ON, Canada
| |
Collapse
|
31
|
Alzahrani AS, Alqahtani A, Abdulkader S, Alluhabi MA, Alqabbas R. Perceived Communication Skills Among Tertiary Care Physicians. MEDICAL SCIENCE EDUCATOR 2019; 29:771-777. [PMID: 34457541 PMCID: PMC8368527 DOI: 10.1007/s40670-019-00764-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
OBJECTIVES Self-perception in clinical skills, including communication skills (CS), has been found to provide insights on strengths, weaknesses, and opportunities for skill improvement. The present study is aimed at exploring perceived CS among physicians working at a tertiary healthcare level. METHODS Physicians working at a tertiary hospital based in Saudi Arabia were invited to complete a modified self-questionnaire that assesses CS. Descriptive and association studies were performed. Psychometric properties of the questionnaire were determined. RESULTS Out of the 101 participating physicians, 57.2% rated their CS in the range of very good and excellent, but only 30.7% rated themselves as overall excellent. The question item with the highest mean score (score range, 1 to 5) was related to encouraging patients to ask questions (4.2 ± 0.9), while the lowest was for the item that assessed information disclosure (3.8 ± 0.8). Males rated themselves higher than females in the item related to explaining things to patients (p < 0.05), whereas physicians with non-surgical specialties rated themselves better than those with surgical specialties in the item related to expressing interest in patients (p < 0.05). CONCLUSIONS Based on physicians' self-rated assessment, less than the third of tertiary care physicians considered themselves as maintaining an excellent level of CS. Future studies are encouraged to examine CS through a multisystem assessment and promote the need for CS training for physicians working at a tertiary care level.
Collapse
Affiliation(s)
- Ahmad S Alzahrani
- Mental Health Department, Neuroscience Center, King Abdullah Medical City, Makkah, Saudi Arabia
- Neuroscience Department, King Faisal Specialist Hospital and Research Center, Jeddah, Saudi Arabia
| | - Abdullah Alqahtani
- Mental Health Department, Neuroscience Center, King Abdullah Medical City, Makkah, Saudi Arabia
| | - Sayed Abdulkader
- Neuroscience Department, King Faisal Specialist Hospital and Research Center, Jeddah, Saudi Arabia
| | | | | |
Collapse
|
32
|
Brookes O, Brown C, Tarrant C, Archer J, Buckley D, Buckley LM, Clement I, Evison F, Gao Smith F, Gibbins C, Hayton E, Jones J, Lilford R, Mullhi R, Packer G, Perkins G, Shelton J, Snelson C, Sullivan P, Vlaev I, Wolstenholme D, Wright SE, Bion J. Patient experience and reflective learning (PEARL): a mixed methods protocol for staff insight development in acute and intensive care medicine in the UK. BMJ Open 2019; 9:e030679. [PMID: 31345985 PMCID: PMC6661565 DOI: 10.1136/bmjopen-2019-030679] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/26/2019] [Revised: 06/19/2019] [Accepted: 06/25/2019] [Indexed: 12/30/2022] Open
Abstract
INTRODUCTION Patient and staff experiences are strongly influenced by attitudes and behaviours, and provide important insights into care quality. Patient and staff feedback could be used more effectively to enhance behaviours and improve care through systematic integration with techniques for reflective learning. We aim to develop a reflective learning framework and toolkit for healthcare staff to improve patient, family and staff experience. METHODS & ANALYSIS Local project teams including staff and patients from the acute medical units (AMUs) and intensive care units (ICUs) of three National Health Service trusts will implement two experience surveys derived from existing instruments: a continuous patient and relative survey and an annual staff survey. Survey data will be supplemented by ethnographic interviews and observations in the workplace to evaluate barriers to and facilitators of reflective learning. Using facilitated iterative co-design, local project teams will supplement survey data with their experiences of healthcare to identify events, actions, activities and interventions which promote personal insight and empathy through reflective learning. Outputs will be collated by the central project team to develop a reflective learning framework and toolkit which will be fed back to the local groups for review, refinement and piloting. The development process will be mapped to a conceptual theory of reflective learning which combines psychological and pedagogical theories of learning, alongside theories of behaviour change based on capability, opportunity and motivation influencing behaviour. The output will be a locally-adaptable workplace-based toolkit providing guidance on using reflective learning to incorporate patient and staff experience in routine clinical activities. ETHICS & DISSEMINATION The PEARL project has received ethics approval from the London Brent Research Ethics Committee (REC Ref 16/LO/224). We propose a national cluster randomised step-wedge trial of the toolkit developed for large-scale evaluation of impact on patient outcomes.
Collapse
Affiliation(s)
- Olivia Brookes
- Research, Development & Innovation, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
| | - Celia Brown
- Warwick Medical School (WMS), The University of Warwick, Warwick, UK
| | | | - Julian Archer
- Royal Australasian College of Surgeons, Melbourne, Victoria, Australia
- Peninsula College of Medicine and Dentistry, Plymouth, UK
| | | | | | - Ian Clement
- Newcastle Upon Tyne Hospitals NHS Foundation Trust, Newcastle Upon Tyne, UK
| | - Felicity Evison
- Department of Health Informatics, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
| | - Fang Gao Smith
- Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
- Academic Department of Anaesthesia, Critical Care, Pain and Resuscitation, Heart of England NHS Foundation Trust, Birmingham, UK
| | - Chris Gibbins
- Newcastle Upon Tyne Hospitals NHS Foundation Trust, Newcastle Upon Tyne, UK
| | - Emma Hayton
- University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
| | | | - Richard Lilford
- Division of Health and Population Sciences, University of Warwick, Coventry, UK
| | - Randeep Mullhi
- University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
| | - Greg Packer
- University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
| | - Gavin Perkins
- Clinical Trials Unit, University of Warwick, Birmingham, UK
| | - Jonathan Shelton
- Newcastle Upon Tyne Hospitals NHS Foundation Trust, Newcastle Upon Tyne, UK
| | | | | | - Ivo Vlaev
- Warwick Business School, Coventry, UK
| | | | - Stephen E Wright
- Anaesthesia, Newcastle upon Tyne Hospitals, Newcastle upon Tyne, UK
| | - Julian Bion
- Intensive Care Medicine, University of Birmingham, Birmingham, UK
| |
Collapse
|
33
|
McGinness AK, Wamsley M, Rivera J. Assessing interprofessional collaboration: Pilot of an interprofessional feedback survey for first-year medical students. ACTA ACUST UNITED AC 2019. [DOI: 10.1016/j.xjep.2019.03.012] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
|
34
|
Simons MR, Zurynski Y, Cullis J, Morgan MK, Davidson AS. Does evidence-based medicine training improve doctors' knowledge, practice and patient outcomes? A systematic review of the evidence. MEDICAL TEACHER 2019; 41:532-538. [PMID: 30328793 DOI: 10.1080/0142159x.2018.1503646] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/26/2023]
Abstract
Courses in Evidence-Based Medicine (EBM) for doctors have consistently demonstrated short-term improvements in knowledge. However, there is no strong evidence linking EBM training to changes in clinical practice or patient outcomes. This systematic review investigates whether EBM training leads to sustained improvements in doctors' knowledge and practice behaviors that may also facilitate changes in patient outcomes and experiences. A literature search was undertaken in Ovid Medline, Ovid Embase, The Cochrane Library, ERIC and Scopus. Studies published from 1997 to 2016 that assessed outcomes of EBM educational interventions amongst doctors and used measures of knowledge, skills, attitudes, practice or patient outcomes were included. Fifteen studies were included in the analysis: four randomized controlled trials (RCTs), three non-RCTs, and eight before-after (longitudinal cohort) studies. Heterogeneity among studies prevented meaningful comparisons. Varying degrees of bias due to the use of subjective measures were identified, limiting study validity. Results showed that EBM interventions can improve short-term knowledge and skills, but there is little reliable evidence of changes in long-term knowledge, attitudes, and clinical practice. No study measured improvement in patient outcomes or experiences. EBM training for medical practitioners needs to incorporate measures of behavioral changes while incorporating patient outcomes and experience measures.
Collapse
Affiliation(s)
- Mary R Simons
- a Faculty of Medicine and Health Sciences , Macquarie University , Sydney , Australia
| | - Yvonne Zurynski
- a Faculty of Medicine and Health Sciences , Macquarie University , Sydney , Australia
| | - Jeremy Cullis
- a Faculty of Medicine and Health Sciences , Macquarie University , Sydney , Australia
| | - Michael K Morgan
- a Faculty of Medicine and Health Sciences , Macquarie University , Sydney , Australia
| | - Andrew S Davidson
- a Faculty of Medicine and Health Sciences , Macquarie University , Sydney , Australia
| |
Collapse
|
35
|
Moreau KA, Eady K, Jabbour M. Exploring residents' reactions to and use of parent feedback in a pediatric emergency department: A grounded theory study. MEDICAL TEACHER 2019; 41:207-214. [PMID: 29688106 DOI: 10.1080/0142159x.2018.1460658] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
BACKGROUND Competency-based medical education (CBME) involves workplace-based assessment. In pediatrics, patients' parents can participate in this assessment and generate feedback for residents. Prior to routinely collecting parent feedback, it is important to investigate residents' perspectives on it. AIM To explore residents' reactions to and use of written parent feedback. METHODS Using a grounded theory approach, we interviewed residents who received written parent feedback at the mid- and end-points of a pediatric emergency training rotation. RESULTS Twenty-five residents participated. The residents reacted positively to the feedback. They thought that it complements educators' feedback, can elucidate parents' perspectives and needs, and is something that residents want and need. Although the residents thought that non-specific negative parent feedback is not useful, they believed non-specific positive and constructive parent feedback to be encouraging and useful. They delineated how they use non-specific positive parent feedback to boost their self-confidence and reassure themselves that parents perceive their clinical practices as appropriate. They also elucidated how they use constructive parent feedback to understand what is important to parents, become aware of their own behaviors, and modify their clinical practices. CONCLUSIONS The findings encourage educators in pediatrics to include parents in resident assessment, especially in the CBME era.
Collapse
Affiliation(s)
- Katherine A Moreau
- a Faculty of Education , University of Ottawa , Ottawa , Canada
- b Children's Hospital of Eastern Ontario Research Institute , Ottawa , Canada
| | - Kaylee Eady
- b Children's Hospital of Eastern Ontario Research Institute , Ottawa , Canada
| | - Mona Jabbour
- c Department of Emergency Medicine, Faculty of Medicine , University of Ottawa , Ottawa , Canada
- d Department of Pediatrics, Faculty of Medicine , University of Ottawa , Ottawa , Canada
- e Children's Hospital of Eastern Ontario , Ottawa , Canada
| |
Collapse
|
36
|
Pooley M, Pizzuti C, Daly M. Optimizing Multisource Feedback Implementation for Australasian Physicians. THE JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS 2019; 39:228-235. [PMID: 31567831 DOI: 10.1097/ceh.0000000000000267] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
INTRODUCTION Medical regulatory bodies in Australasia are encouraging greater use of continuing professional development activities, such as multisource feedback (MSF), which are practice-based, include facilitated feedback, and improve performance. The aim of this study was to explore the feasibility, effectiveness, and sustainability of an MSF process that includes a telephone/videoconference debrief, to better design future MSF implementation. METHOD Thirty-seven Australasian physician participants sought feedback from patients and colleagues and debriefed their feedback report with a trained facilitator. The impact was evaluated using quantitative and qualitative measures including surveys and semistructured interviews. RESULTS The feasibility of the MSF process was impacted by the level of support for the participant to complete the MSF, the nature and location of participants' work, and the use of telephone/videoconference to deliver the debrief. Regarding effectiveness, the MSF stimulated reflection on practice and action on areas identified for improvement. In addition, the quality of rater feedback and the inclusion of a debrief impacted participant and facilitator reports of effectiveness. The absence of a culture of feedback, the formative nature of the MSF, and the confidentiality of the results were factors impacting sustainability. DISCUSSION Optimizing MSF is important to encourage broad uptake in the wider medical community in Australasia. Although several factors were identified as having an impact, it is clear that inclusion of a quality debrief will increase the perceived value and the effectiveness of MSF. Delivering that debrief through telephone/videoconference can be effective and will increase the overall feasibility and sustainability.
Collapse
Affiliation(s)
- Michael Pooley
- Mr. Pooley: Senior Project Officer, Directorate of Professional Practice, RACP, Sydney, New South Wales, Australia. Ms. Pizzuti: Research Officer, Directorate of Professional Practice, RACP, Sydney, New South Wales, Australia. Ms. Daly: Senior Researcher, Directorate of Professional Practice, RACP, Sydney, New South Wales, Australia. Honorary Research Fellow, School of Public Health, University of Sydney, Sydney, New South Wales, Australia
| | | | | |
Collapse
|
37
|
Baines R, Regan de Bere S, Stevens S, Read J, Marshall M, Lalani M, Bryce M, Archer J. The impact of patient feedback on the medical performance of qualified doctors: a systematic review. BMC MEDICAL EDUCATION 2018; 18:173. [PMID: 30064413 PMCID: PMC6069829 DOI: 10.1186/s12909-018-1277-0] [Citation(s) in RCA: 38] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/27/2017] [Accepted: 07/11/2018] [Indexed: 05/21/2023]
Abstract
BACKGROUND Patient feedback is considered integral to quality improvement and professional development. However, while popular across the educational continuum, evidence to support its efficacy in facilitating positive behaviour change in a postgraduate setting remains unclear. This review therefore aims to explore the evidence that supports, or refutes, the impact of patient feedback on the medical performance of qualified doctors. METHODS Electronic databases PubMed, EMBASE, Medline and PsycINFO were systematically searched for studies assessing the impact of patient feedback on medical performance published in the English language between 2006-2016. Impact was defined as a measured change in behaviour using Barr's (2000) adaptation of Kirkpatrick's four level evaluation model. Papers were quality appraised, thematically analysed and synthesised using a narrative approach. RESULTS From 1,269 initial studies, 20 articles were included (qualitative (n=8); observational (n=6); systematic review (n=3); mixed methodology (n=1); randomised control trial (n=1); and longitudinal (n=1) design). One article identified change at an organisational level (Kirkpatrick level 4); six reported a measured change in behaviour (Kirkpatrick level 3b); 12 identified self-reported change or intention to change (Kirkpatrick level 3a), and one identified knowledge or skill acquisition (Kirkpatrick level 2). No study identified a change at the highest level, an improvement in the health and wellbeing of patients. The main factors found to influence the impact of patient feedback were: specificity; perceived credibility; congruence with physician self-perceptions and performance expectations; presence of facilitation and reflection; and inclusion of narrative comments. The quality of feedback facilitation and local professional cultures also appeared integral to positive behaviour change. CONCLUSION Patient feedback can have an impact on medical performance. However, actionable change is influenced by several contextual factors and cannot simply be guaranteed. Patient feedback is likely to be more influential if it is specific, collected through credible methods and contains narrative information. Data obtained should be fed back in a way that facilitates reflective discussion and encourages the formulation of actionable behaviour change. A supportive cultural understanding of patient feedback and its intended purpose is also essential for its effective use.
Collapse
Affiliation(s)
- Rebecca Baines
- Collaboration for the Advancement of Medical Education Research & Assessment (CAMERA), Faculty of Medicine and Dentistry, University of Plymouth, Drake Circus, Plymouth, PL4 8AA, UK
| | - Sam Regan de Bere
- Collaboration for the Advancement of Medical Education Research & Assessment (CAMERA), Faculty of Medicine and Dentistry, University of Plymouth, Drake Circus, Plymouth, PL4 8AA, UK
| | - Sebastian Stevens
- Collaboration for the Advancement of Medical Education Research & Assessment (CAMERA), Faculty of Medicine and Dentistry, University of Plymouth, Drake Circus, Plymouth, PL4 8AA, UK
| | - Jamie Read
- Collaboration for the Advancement of Medical Education Research & Assessment (CAMERA), Faculty of Medicine and Dentistry, University of Plymouth, Drake Circus, Plymouth, PL4 8AA, UK
| | - Martin Marshall
- Improvement Science London, University College London, London, UK
| | - Mirza Lalani
- Research Department of Primary Care and Population Health, University College London, London, UK
| | - Marie Bryce
- Collaboration for the Advancement of Medical Education Research & Assessment (CAMERA), Faculty of Medicine and Dentistry, University of Plymouth, Drake Circus, Plymouth, PL4 8AA, UK
| | - Julian Archer
- Collaboration for the Advancement of Medical Education Research & Assessment (CAMERA), Faculty of Medicine and Dentistry, University of Plymouth, Drake Circus, Plymouth, PL4 8AA, UK.
| |
Collapse
|
38
|
Yama BA, Hodgins M, Boydell K, Schwartz SB. A qualitative exploration: questioning multisource feedback in residency education. BMC MEDICAL EDUCATION 2018; 18:170. [PMID: 30041635 PMCID: PMC6057061 DOI: 10.1186/s12909-018-1270-7] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/30/2017] [Accepted: 06/27/2018] [Indexed: 06/01/2023]
Abstract
BACKGROUND Multisource feedback (MSF), involves the collection of feedback from multiple groups of assessors, including those without a traditional hierarchal responsibility to evaluate doctors. Allied healthcare professionals (AHCPs), administrative staff, peers, patients and their families may all contribute to the formative assessment of physicians. Theoretically, this feedback provides a thorough view of physician performance; however, the ability of MSF programs to consistently impact physician behavior remains in question. Therefore, the objective of this study was to explore perceptions and prerequisites to an effective MSF program in postgraduate medical education from the perspectives of both pediatric residents and AHCPs. METHODS This exploratory study was conducted in a pediatric inpatient unit prior to implementation of a MSF program. Focus groups were conducted with purposefully recruited participants from three distinct groups: junior pediatric residents, senior pediatric residents, and AHCPs. Discussions were audio recorded, transcribed verbatim and analyzed using thematic analysis. RESULTS Both residents and AHCPs expressed a strong interest in the concept of MSF. However, more in depth discussions identified barriers to residents' acceptance of, and AHCPs' provision of feedback. Roles and responsibilities, perceptions of expertise, hospital culture/interprofessionalism and power dynamics were identified as barriers to the acceptance and provision of feedback. All groups expressed interest in opportunities to engage in bi-directional feedback. CONCLUSIONS The identified barriers and prerequisites to providing and accepting MSF suggest limits to the efficacy of the MSF process. Our findings suggest that these factors should be considered in the design and implementation of MSF programs.
Collapse
Affiliation(s)
- Brie A. Yama
- Department of Paediatrics, The University of Toronto, Toronto, ON Canada
- Division of Paediatric Medicine, The Hospital for Sick Children, Toronto, ON Canada
| | | | - Katherine Boydell
- Black Dog Institute, University of New South Wales, Randwick, NSW Australia
| | - Sarah B. Schwartz
- Department of Paediatrics, The University of Toronto, Toronto, ON Canada
- Division of Paediatric Medicine, The Hospital for Sick Children, Toronto, ON Canada
| |
Collapse
|
39
|
Weenink JW, Kool RB, Hesselink G, Bartels RH, Westert GP. Prevention of and dealing with poor performance: an interview study about how professional associations aim to support healthcare professionals. Int J Qual Health Care 2018; 29:838-844. [PMID: 29024984 DOI: 10.1093/intqhc/mzx114] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/19/2017] [Accepted: 08/17/2017] [Indexed: 11/14/2022] Open
Abstract
Objective To explore how professional associations of nine healthcare professions aim to support professionals to prevent and deal with poor performance. Design Qualitative interview study. Setting The Netherlands. Participants Representatives of professional associations for dentists, general practitioners, medical specialists, midwives, nurses, pharmacists, physiotherapists, psychologists and psychotherapists. Interventions During nine face-to-face semi-structured interviews we asked how associations aim to support professionals in prevention of and dealing with poor performance. Following the first interview, we monitored new initiatives in support over a 2.5-year period, after which we conducted a second interview. Interviews were analysed using thematic analysis. Main outcome measures Available policy and support regarding poor performance. Results Three themes emerged from our data (i.e. elaborating on professional performance, performance insight and dealing with poor performance) for which we identified a total of 10 categories of support. Support concerned professional codes, guidelines and codes of conduct, quality registers, individual performance assessment, peer consultation, practice evaluation, helpdesk and expert counselling, a protocol for dealing with poor performance, a place for support and to report poor performance, and internal disciplinary procedures. Conclusions This study provides an overview of support given to nine healthcare professions by their associations regarding poor performance, and identifies gaps that associations could follow up on, such as clarifying what to do when confronted with a poorly performing colleague, supporting professionals that poorly perform, and developing methods for individual performance assessment to gain performance insight. A next step would be to evaluate the use and effect of different types of support.
Collapse
Affiliation(s)
- Jan-Willem Weenink
- Scientific Center for Quality of Healthcare (IQ healthcare), Radboud Institute for Health Sciences, Radboud University Medical Center, PO Box 9101, 6500 HB Nijmegen, Netherlands.,Institute of Health Policy & Management, Erasmus University Rotterdam, PO Box 1738, 3000 DR Rotterdam, Netherlands
| | - Rudolf B Kool
- Scientific Center for Quality of Healthcare (IQ healthcare), Radboud Institute for Health Sciences, Radboud University Medical Center, PO Box 9101, 6500 HB Nijmegen, Netherlands
| | - Gijs Hesselink
- Scientific Center for Quality of Healthcare (IQ healthcare), Radboud Institute for Health Sciences, Radboud University Medical Center, PO Box 9101, 6500 HB Nijmegen, Netherlands
| | - Ronald H Bartels
- Department of Neurosurgery, Radboud University Medical Center, PO Box 9101, 6500 HB Nijmegen, Netherlands
| | - Gert P Westert
- Scientific Center for Quality of Healthcare (IQ healthcare), Radboud Institute for Health Sciences, Radboud University Medical Center, PO Box 9101, 6500 HB Nijmegen, Netherlands
| |
Collapse
|
40
|
Burt J, Abel G, Elliott MN, Elmore N, Newbould J, Davey A, Llanwarne N, Maramba I, Paddison C, Campbell J, Roland M. The Evaluation of Physicians' Communication Skills From Multiple Perspectives. Ann Fam Med 2018; 16:330-337. [PMID: 29987081 PMCID: PMC6037531 DOI: 10.1370/afm.2241] [Citation(s) in RCA: 25] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/01/2017] [Revised: 01/30/2018] [Accepted: 02/27/2018] [Indexed: 11/09/2022] Open
Abstract
PURPOSE To examine how family physicians', patients', and trained clinical raters' assessments of physician-patient communication compare by analysis of individual appointments. METHODS Analysis of survey data from patients attending face-to-face appointments with 45 family physicians at 13 practices in England. Immediately post-appointment, patients and physicians independently completed a questionnaire including 7 items assessing communication quality. A sample of videotaped appointments was assessed by trained clinical raters, using the same 7 communication items. Patient, physician, and rater communication scores were compared using correlation coefficients. RESULTS Included were 503 physician-patient pairs; of those, 55 appointments were also evaluated by trained clinical raters. Physicians scored themselves, on average, lower than patients (mean physician score 74.5; mean patient score 94.4); 63.4% (319) of patient-reported scores were the maximum of 100. The mean of rater scores from 55 appointments was 57.3. There was a near-zero correlation coefficient between physician-reported and patient-reported communication scores (0.009, P = .854), and between physician-reported and trained rater-reported communication scores (-0.006, P = .69). There was a moderate and statistically significant association, however, between patient and trained-rater scores (0.35, P = .042). CONCLUSIONS The lack of correlation between physician scores and those of others indicates that physicians' perceptions of good communication during their appointments may differ from those of external peer raters and patients. Physicians may not be aware of how patients experience their communication practices; peer assessment of communication skills is an important approach in identifying areas for improvement.
Collapse
Affiliation(s)
- Jenni Burt
- The Healthcare Improvement Studies Institute (THIS Institute), University of Cambridge, Cambridge Biomedical Campus, Cambridge, United Kingdom
| | - Gary Abel
- University of Exeter Medical School, St Luke's Campus, Exeter, United Kingdom
| | | | - Natasha Elmore
- The Healthcare Improvement Studies Institute (THIS Institute), University of Cambridge, Cambridge Biomedical Campus, Cambridge, United Kingdom
| | | | - Antoinette Davey
- University of Exeter Medical School, St Luke's Campus, Exeter, United Kingdom
| | - Nadia Llanwarne
- Cambridge Centre for Health Services Research, University of Cambridge School of Clinical Medicine, Cambridge, United Kingdom
| | - Inocencio Maramba
- University of Exeter Medical School, St Luke's Campus, Exeter, United Kingdom
| | - Charlotte Paddison
- Cambridge Centre for Health Services Research, University of Cambridge School of Clinical Medicine, Cambridge, United Kingdom
| | - John Campbell
- University of Exeter Medical School, St Luke's Campus, Exeter, United Kingdom
| | - Martin Roland
- Cambridge Centre for Health Services Research, University of Cambridge School of Clinical Medicine, Cambridge, United Kingdom
| |
Collapse
|
41
|
Sehlbach C, Govaerts MJ, Mitchell S, Rohde GGU, Smeenk FWJM, Driessen EW. Doctors on the move: a European case study on the key characteristics of national recertification systems. BMJ Open 2018; 8:e019963. [PMID: 29666131 PMCID: PMC5905769 DOI: 10.1136/bmjopen-2017-019963] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/05/2017] [Revised: 03/09/2018] [Accepted: 03/19/2018] [Indexed: 01/02/2023] Open
Abstract
OBJECTIVES With increased cross-border movement, ensuring safe and high-quality healthcare has gained primacy. The purpose of recertification is to ensure quality of care through periodically attesting doctors' professional proficiency in their field. Professional migration and facilitated cross-border recognition of qualifications, however, make us question the fitness of national policies for safeguarding patient care and the international accountability of doctors. DESIGN AND SETTING We performed document analyses and conducted 19 semistructured interviews to identify and describe key characteristics and effective components of 10 different European recertification systems, each representing one case (collective case study). We subsequently compared these systems to explore similarities and differences in terms of assessment criteria used to determine process quality. RESULTS Great variety existed between countries in terms and assessment formats used, targeting cognition, competence and performance (Miller's assessment pyramid). Recertification procedures and requirements also varied significantly, ranging from voluntary participation in professional development modules to the mandatory collection of multiple performance data in a competency-based portfolio. Knowledge assessment was fundamental to recertification in most countries. Another difference concerned the stakeholders involved in the recertification process: while some systems exclusively relied on doctors' self-assessment, others involved multiple stakeholders but rarely included patients in assessment of doctors' professional competence. Differences between systems partly reflected different goals and primary purposes of recertification. CONCLUSION Recertification systems differ substantially internationally with regard to the criteria they apply to assess doctors' competence, their aims, requirements, assessment formats and patient involvement. In the light of professional mobility and associated demands for accountability, we recommend that competence assessment includes patients' perspectives, and recertification practices be shared internationally to enhance transparency. This can help facilitate cross-border movement, while guaranteeing high-quality patient care.
Collapse
Affiliation(s)
- Carolin Sehlbach
- Department of Educational Development and Research, SHE, Maastricht University, Maastricht, The Netherlands
| | - Marjan J Govaerts
- Department of Educational Development and Research, SHE, Maastricht University, Maastricht, The Netherlands
| | - Sharon Mitchell
- Department of Education, European Respiratory Society, Lausanne, Switzerland
| | - Gernot G U Rohde
- Respiratory Medicine, Johann Wolfgang Goethe University Hospital, Frankfurt, Germany
| | - Frank W J M Smeenk
- Department of Educational Development and Research, SHE, Maastricht University, Maastricht, The Netherlands
- Respiratory Medicine, Catharina Hospital, Eindhoven, The Netherlands
| | - Erik W Driessen
- Department of Educational Development and Research, SHE, Maastricht University, Maastricht, The Netherlands
| |
Collapse
|
42
|
Bogetz AL. How Residents Learn From Patient Feedback: A Multi-Institutional Qualitative Study of Pediatrics Residents' Perspectives. J Grad Med Educ 2018; 10:176-184. [PMID: 29686757 PMCID: PMC5901797 DOI: 10.4300/jgme-d-17-00447.1] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/27/2017] [Revised: 10/16/2017] [Accepted: 12/13/2017] [Indexed: 11/06/2022] Open
Abstract
BACKGROUND Residents may view feedback from patients and their families with greater skepticism than feedback from supervisors and peers. While discussing patient and family feedback with faculty may improve residents' acceptance of feedback and learning, specific strategies have not been identified. OBJECTIVE We explored pediatrics residents' perspectives of patient feedback and identified strategies that promote residents' reflection on and learning from feedback. METHODS In this multi-institutional, qualitative study conducted in June and July 2016, we conducted focus groups with a purposive sample of pediatrics residents after their participation in a randomized controlled trial in which they received written patient feedback and either discussed it with faculty or reviewed it independently. Focus group transcripts were audiorecorded, transcribed, and analyzed for themes using the constant comparative approach associated with grounded theory. RESULTS Thirty-six of 92 (39%) residents participated in 7 focus groups. Four themes emerged: (1) residents valued patient feedback but felt it may lack the specificity they desire; (2) discussing feedback with a trusted faculty member was helpful for self-reflection; (3) residents identified 5 strategies faculty used to facilitate their openness to and acceptance of patient feedback (eg, help resident overcome emotional responses to feedback and situate feedback in the context of lifelong learning); and (4) residents' perceptions of feedback credibility improved when faculty observed patient encounters and solicited feedback on the resident's behalf prior to discussions. CONCLUSIONS Discussing patient feedback with faculty provided important scaffolding to enhance residents' openness to and reflection on patient feedback.
Collapse
|
43
|
Eady K, Moreau KA. Using parent feedback: A qualitative study of residents' and physician-educators' perspectives. PERSPECTIVES ON MEDICAL EDUCATION 2018; 7:33-39. [PMID: 29256055 PMCID: PMC5807261 DOI: 10.1007/s40037-017-0393-6] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/01/2023]
Abstract
INTRODUCTION Patients and family members can contribute to resident assessment in competency-based medical education. However, few studies have examined the use of patient/family member feedback generated from questionnaire-based assessments. To implement appropriate assessment strategies and optimize feedback use, we need to understand how residents and physician-educators would use feedback from these stakeholders. This study aimed to understand how paediatric residents and physician-educators would use parent feedback generated from questionnaire-based assessments. METHODS This study was conducted at a paediatric academic health science centre. We held dyadic interviews with six residents and six physician-educators. A three-step approach was used to analyze the data: data reduction, data display, and conclusions/verifications. We developed an initial coding scheme, conducted an in-depth review of the data and coded it, finalized our coding scheme, and identified categories. RESULTS Participants described that they would use parent feedback to: (a) provide additional direct observations of residents' performances, (b) teach and coach residents, (c) assess residents' overall performance and progression, and (d) encourage resident self-assessment and behaviour change. DISCUSSION Parents directly observe residents as they interact with them and their children and, therefore, can provide feedback on residents' performances. Residency programs should include parent feedback and promote and facilitate its use by residents and physician-educators. CONCLUSION This study provides an initial understanding of how paediatric residents and physician-educators would use parent feedback if they were to receive it. This information, combined with future research, can inform the development and implementation of parent feedback strategies in competency-based medical education.
Collapse
Affiliation(s)
- Kaylee Eady
- Children's Hospital of Eastern Ontario Research Institute, Ottawa, Ontario, Canada.
| | | |
Collapse
|
44
|
Lockyer JM, Sargeant J, Richards SH, Campbell JL, Rivera LA. Multisource Feedback and Narrative Comments: Polarity, Specificity, Actionability, and CanMEDS Roles. THE JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS 2018; 38:32-40. [PMID: 29329147 DOI: 10.1097/ceh.0000000000000183] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/07/2023]
Abstract
INTRODUCTION Multisource feedback is a questionnaire-based assessment tool that provides physicians with data about workplace behaviors and may combine numeric and narrative (free-text) comments. Little attention has been paid to wording of requests for comments, potentially limiting its utility to support physician performance. This study tested the phrasing of two different sets of questions. METHODS Two sets of questions were tested with family physicians, medical and surgical specialists, and their medical colleague and coworker respondents. One set asked respondents to identify one thing the participant physician does well and one thing the physician could target for action. Set 2 questions asked what does the physician do well and what might the physician do to enhance practice. Resulting free-text comments provided by respondents were coded for polarity (positive, neutral, or negative), specificity (precision and detail), actionability (ability to use the feedback to direct future activity), and CanMEDS roles (competencies) and analyzed descriptively. RESULTS Data for 222 physicians (111 physicians per set) were analyzed. A total of 1824 comments (8.2/physician) were submitted, with more comments from coworkers than medical colleagues. Set 1 yielded more comments and were more likely to be positive, semi specific, and very actionable than set 2. However, set 2 generated more very specific comments. Comments covered all CanMEDS roles with more comments for collaborator and leader roles. DISCUSSION The wording of questions inviting free-text responses influences the volume and nature of the comments provided. Individuals designing multisource feedback tools should carefully consider wording of items soliciting narrative responses.
Collapse
Affiliation(s)
- Jocelyn M Lockyer
- Dr. Lockyer: Professor, Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada. Dr. Sargeant: Professor, Medical Education, Dalhousie University, Halifax, Nova Scotia, Canada. Dr. Richards: Academic Unit of Primary Care, Leeds Institute of Health Sciences, University of Leeds, Leeds, United Kingdom. Dr. Campbell: Professor of General Practice and Primary Care and Director, University of Exeter Collaboration for Academic Primary Care (APEx), University of Exeter Medical School, Exeter, United Kingdom. Ms. Rivera: Research Associate, Office of Continuing Medical Education and Professional Development, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada
| | | | | | | | | |
Collapse
|
45
|
Meidani Z, Farzandipour M, Davoodabadi A, Farrokhian A, Kheirkhah D, Sharifi M, Khanghahi ME. Effect of reinforced audit and feedback intervention on physician behaviour: a multifaceted strategy for targeting medical record documentation. J R Coll Physicians Edinb 2018; 47:237-242. [DOI: 10.4997/jrcpe.2017.305] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022] Open
|
46
|
Stevens S, Read J, Baines R, Chatterjee A, Archer J. Validation of Multisource Feedback in Assessing Medical Performance: A Systematic Review. THE JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS 2018; 38:262-268. [PMID: 30157152 DOI: 10.1097/ceh.0000000000000219] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
INTRODUCTION Over the past 10 years, a number of systematic reviews have evaluated the validity of multisource feedback (MSF) to assess and quality-assure medical practice. The purpose of this study is to synthesize the results from existing reviews to provide a holistic overview of the validity evidence. METHODS This review identified eight systematic reviews evaluating the validity of MSF published between January 2006 and October 2016. Using a standardized data extraction form, two independent reviewers extracted study characteristics. A framework of validation developed by the American Psychological Association was used to appraise the validity evidence within each systematic review. RESULTS In terms of validity evidence, each of the eight reviews demonstrated evidence across at least one domain of the American Psychological Association's validity framework. Evidence of assessment validity within the domains of "internal structure" and "relationship to other variables" has been well established. However, the domains of content validity (ie, ensuring that MSF tools measure what they are intended to measure); consequential validity (ie, evidence of the intended or unintended consequences MSF assessments may have on participants or wider society), and response process validity (ie, the process of standardization and quality control in the delivery and completion of assessments) remain limited. DISCUSSION Evidence for the validity of MSF has, across a number of domains, been well established. However, the size and quality of the existing evidence remains variable. To determine the extent to which MSF is considered a valid instrument to assess medical performance, future research is required to determine the following: (1) how best to design and deliver MSF assessments that address the identified limitations of existing tools and (2) how to ensure that involvement within MSF supports positive changes in practice. Such research is integral if MSF is to continue to inform medical performance and subsequent improvements in the quality and safety of patient care.
Collapse
Affiliation(s)
- Sebastian Stevens
- Collaboration for the Advancement of Medical Education Research & Assessment (CAMERA), Plymouth University Peninsula Schools of Medicine and Dentistry (PU PSMD), University of Plymouth, Plymouth, PL, UK
| | | | | | | | | |
Collapse
|
47
|
Field JC, Cowpe JG, Walmsley AD. The Graduating European Dentist: A New Undergraduate Curriculum Framework. EUROPEAN JOURNAL OF DENTAL EDUCATION : OFFICIAL JOURNAL OF THE ASSOCIATION FOR DENTAL EDUCATION IN EUROPE 2017; 21 Suppl 1:2-10. [PMID: 29205782 DOI: 10.1111/eje.12307] [Citation(s) in RCA: 109] [Impact Index Per Article: 15.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 07/11/2017] [Indexed: 05/05/2023]
Abstract
With "The Graduating European Dentist", ADEE provides a new approach that reflects best academic practice for European undergraduate dental education. The new suite of documents sees increased emphasis on important curriculum components such as patient safety, working as a team and patient-centred care. There is also an increased emphasis on teaching excellence, student satisfaction and student preparedness and support. Guidance relating to methods of teaching and learning, and assessment (educational measurement), is also provided. It is anticipated that this new format will further increase accessibility and utility for educators, institutions, societies and regulators, across the sector.
Collapse
Affiliation(s)
- J C Field
- The University of Sheffield, Sheffield, UK
| | | | | |
Collapse
|
48
|
Sureda-Demeulemeester E, Ramis-Palmer C, Sesé-Abad A. The assessment of medical competencies. Rev Clin Esp 2017. [DOI: 10.1016/j.rceng.2017.05.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
49
|
Nimmo S. Revalidation, appraisal and multisource feedback for occupational physicians. Occup Med (Lond) 2017; 67:413-415. [DOI: 10.1093/occmed/kqx062] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022] Open
|
50
|
Sureda-Demeulemeester E, Ramis-Palmer C, Sesé-Abad A. The assessment of medical competencies. Rev Clin Esp 2017; 217:534-542. [PMID: 28666528 DOI: 10.1016/j.rce.2017.05.004] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2017] [Revised: 05/22/2017] [Accepted: 05/28/2017] [Indexed: 12/30/2022]
Abstract
OBJECTIVE To describe the most widely used tools in the assessment of medical competencies, analyse their prevalence of use, their advantages and disadvantages and propose an appropriate model for our context. METHODS We conducted a narrative review of articles from MEDLINE, following the PRISM protocol, and analysed a total of 62 articles. RESULTS The assessment of competencies is heterogeneous, especially in the educational and professional settings. The specific and technical competencies acquired during university education are mainly assessed using the objective structured clinical assessment. In the professional setting, core competencies are assessed using the 360° technique. CONCLUSIONS We need a rigorous empiric comparison of the efficiency of the tools according to the type of competency. We propose a competency management model for the «undergraduate/graduate/active professional» continuum, whose goal is to improve training and professional practice and thereby increase the quality of patient care.
Collapse
Affiliation(s)
- E Sureda-Demeulemeester
- Departamento de Desarrollo Organizacional, Hospital Universitario Son Espases, Palma, Mallorca, Islas Baleares, España; Departamento de Psicología, Universitat de les Illes Balears, Palma, Mallorca, Islas Baleares, España
| | - C Ramis-Palmer
- Departamento de Psicología, Universitat de les Illes Balears, Palma, Mallorca, Islas Baleares, España
| | - A Sesé-Abad
- Departamento de Psicología, Universitat de les Illes Balears, Palma, Mallorca, Islas Baleares, España.
| |
Collapse
|