1
|
Shen Z, Zeng X, Li J, Zheng M, Guo J, Yang Y, Liu G, Cao C. Does participation in sports competitions enhance interprofessional teamwork among medical students? Evidence from a medical school curriculum experiment. BMC MEDICAL EDUCATION 2024; 24:821. [PMID: 39080590 PMCID: PMC11289920 DOI: 10.1186/s12909-024-05807-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/14/2024] [Accepted: 07/22/2024] [Indexed: 08/02/2024]
Abstract
BACKGROUND Effective interprofessional teamwork is essential for the efficiency, safety and quality of healthcare system services and requires interprofessional education for medical students. Physical education is a simple and easy way to teach teamwork, which translates into team performance in the work environment. This study was conducted to examine the effectiveness of the physical education competition model, instead of the exams model, for improving teamwork skills among medical students. METHODS A quasiexperimental intervention design was used to measure the effect of a 16-week cheerleading programme on subjects' teamwork skills by completing a teamwork scale comprising four subdimensions, namely, personal characteristics, teamwork, leadership, and conflict management, before the start and at the end of the programme, and by comparing nonwinning to winning students to measure the effect of teamwork skills on team performance. RESULTS A total of 179 students completed the valid baseline and posttest (effective rate = 95.21%). The teamwork scale scores (B M = 4.81, R M = 5.05, p < 0.001) and 4 subdimension scores (personal characteristics p = 0.002, teamwork p = 0.028, leadership p < 0.001, conflict management p < 0.001) were statistically significant. Twenty-two of the 44 items in the scale improved significantly. The differences between students who won the competition and those who did not (N M=4.86, W M=5.14, p<0.01) were statistically significant, with no significant differences in personal characteristics p = 0.183; significant differences in the 3 subdimensions of teamwork p < 0.01, leadership p = 0.024, and conflict management p = 0.037; and a significant increase in 13 out of 44 self-efficacy items on the scale. CONCLUSIONS The "race for exams" physical education programme improved teamwork among medical students, and increased teamwork improved team performance. The "competition instead of examination" physical education programme provides a quantifiable method for improving interprofessional teamwork among medical students.
Collapse
Affiliation(s)
- Zhiling Shen
- Chongqing Medical and Pharmaceutical College, Chongqing, 401331, China
| | - Xinrong Zeng
- College of Public Health, Chongqing Medical University, Chongqing, 400331, China
| | - Jianyu Li
- Division of Sports Science and Physical Education, Tsinghua University, Beijing, 100084, China
| | - Man Zheng
- Division of Sports Science and Physical Education, Tsinghua University, Beijing, 100084, China
| | - Jia Guo
- Division of Sports Science and Physical Education, Tsinghua University, Beijing, 100084, China
| | - Yaming Yang
- College of Physical Education, Taiyuan University of Technology, Shanxi Province, Taiyuan, 030600, China
| | - Guochun Liu
- Division of Sports Science and Physical Education, Tsinghua University, Beijing, 100084, China.
- College of Exercise Medicine, Chongqing Medical University, Chongqing, 400331, China.
| | - Chunmei Cao
- Division of Sports Science and Physical Education, Tsinghua University, Beijing, 100084, China.
| |
Collapse
|
2
|
Anwar A, Yasmeen R, Khan RA. Collaboration cognizance: Development of a self-assessment tool to measure intra-professional collaborative practices (IPCP) in postgraduate medical residents at tertiary care hospitals. BMC MEDICAL EDUCATION 2024; 24:771. [PMID: 39030526 PMCID: PMC11264871 DOI: 10.1186/s12909-024-05759-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/22/2023] [Accepted: 07/09/2024] [Indexed: 07/21/2024]
Abstract
BACKGROUND The ever-evolving healthcare system of today demands physicians who steer their roles as treatment providers, managers and collaborators. Professionals are highly interdependent due to increased complexity of health problems and risk of errors increases with transitions in care. In hospitals, the main workforce is postgraduate residents; therefore, intraprofessional collaboration amongst residents is essential. Several instruments are available to evaluate interprofessional collaboration amongst physicians, nurses and hospital teams but none specifically assessed intra-professional collaborative practices amongst residents working in tertiary care hospitals in multi-disciplinary teams. This study intends to develop and validate an instrument to self-assess intraprofessional collaborative practices in postgraduate residents undergoing residency in multiple specialties at tertiary care hospitals. APPROACH This study on Instrument Development employed mixed method study design executed in two phases. In phase 1, six domains of intraprofessional collaborative practices were identified from literature and 35 items were developed. Fifteen experts participated in qualitative content validation and provided comments. To establish content validity in phase 2, content validity index (CVI) and content clarity average (CCA) were assessed by seventeen experts. Response process validity was established by cognitive interviewing of 5 postgraduate residents. Pilot testing was done on a sample of 407 residents. Cronbach's alpha was determined, and confirmatory factor analysis established construct validity. RESULTS During phase 1, items were modified based on qualitative feedback from 15 experts. In round 2, CVI and CCA were determined based on responses of 17 experts. The items having an I-CVI greater than 0.90 were accepted and six items underwent modifications as their I-CVI fell between 0.78 and 0.90. Similarly, four items with a CCA of less than 2.4 were modified to increase clarity. Cognitive interviews of participants on 30 items resulted in the deletion of 1 item and changes in 5 items. The final instrument had 29 items categorized under six constructs. All items had good factor loadings during CFA, so none was deleted. Cronbach's Alpha α was 0.937. CONCLUSION Intraprofessional collaborative practices in residents is a valid and reliable self-assessment tool comprising 29 items measuring six constructs. It may be used by residents to assess their collaborative practices and incorporated in curricula to help develop collaborative practices and their assessment during training of postgraduate residents.
Collapse
Affiliation(s)
- Ayesha Anwar
- Department of Dermatology, Pak Emirates Military Hospital Rawalpindi, Rawalpindi, Pakistan.
- MHPE Fellow at Riphah International University, Islamabad, Pakistan.
| | - Rahila Yasmeen
- Dean Riphah Academy of Research & Education, Riphah International University, Islamabad, Pakistan
| | - Rehan Ahmed Khan
- Dean Riphah Institute of Assessment, Riphah International University, Islamabad, Pakistan
| |
Collapse
|
3
|
Higham H, Greig P, Crabtree N, Hadjipavlou G, Young D, Vincent C. A study of validity and usability evidence for non-technical skills assessment tools in simulated adult resuscitation scenarios. BMC MEDICAL EDUCATION 2023; 23:153. [PMID: 36906567 PMCID: PMC10007667 DOI: 10.1186/s12909-023-04108-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/01/2022] [Accepted: 02/15/2023] [Indexed: 06/18/2023]
Abstract
BACKGROUND Non-technical skills (NTS) assessment tools are widely used to provide formative and summative assessment for healthcare professionals and there are now many of them. This study has examined three different tools designed for similar settings and gathered evidence to test their validity and usability. METHODS Three NTS assessment tools designed for use in the UK were used by three experienced faculty to review standardized videos of simulated cardiac arrest scenarios: ANTS (Anesthetists' Non-Technical Skills), Oxford NOTECHS (Oxford NOn-TECHnical Skills) and OSCAR (Observational Skill based Clinical Assessment tool for Resuscitation). Internal consistency, interrater reliability and quantitative and qualitative analysis of usability were analyzed for each tool. RESULTS Internal consistency and interrater reliability (IRR) varied considerably for the three tools across NTS categories and elements. Intraclass correlation scores of three expert raters ranged from poor (task management in ANTS [0.26] and situation awareness (SA) in Oxford NOTECHS [0.34]) to very good (problem solving in Oxford NOTECHS [0.81] and cooperation [0.84] and SA [0.87] in OSCAR). Furthermore, different statistical tests of IRR produced different results for each tool. Quantitative and qualitative examination of usability also revealed challenges in using each tool. CONCLUSIONS The lack of standardization of NTS assessment tools and training in their use is unhelpful for healthcare educators and students. Educators require ongoing support in the use of NTS assessment tools for the evaluation of individual healthcare professionals or healthcare teams. Summative or high-stakes examinations using NTS assessment tools should be undertaken with at least two assessors to provide consensus scoring. In light of the renewed focus on simulation as an educational tool to support and enhance training recovery in the aftermath of COVID-19, it is even more important that assessment of these vital skills is standardized, simplified and supported with adequate training.
Collapse
Affiliation(s)
- Helen Higham
- Nuffield Department of Clinical Neurosciences, University of Oxford, Level 6, West Wing, John Radcliffe Hospital, Oxford, OX3 9DU England
| | - Paul Greig
- Nuffield Department of Clinical Neurosciences, University of Oxford, Level 6, West Wing, John Radcliffe Hospital, Oxford, OX3 9DU England
| | - Nick Crabtree
- The Medical Specialist Group LLP Guernsey, Saint Peter Port, Guernsey
| | | | - Duncan Young
- Nuffield Department of Clinical Neurosciences, University of Oxford, Level 6, West Wing, John Radcliffe Hospital, Oxford, OX3 9DU England
| | - Charles Vincent
- Department of Experimental Psychology, University of Oxford, Oxford, UK
| |
Collapse
|
4
|
Shimizu I, Matsuyama Y, Duvivier R, van der Vleuten C. Perceived positive social interdependence in online versus face-to-face team-based learning styles of collaborative learning: a randomized, controlled, mixed-methods study. BMC MEDICAL EDUCATION 2022; 22:567. [PMID: 35869477 PMCID: PMC9307427 DOI: 10.1186/s12909-022-03633-y] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/24/2022] [Accepted: 07/18/2022] [Indexed: 06/15/2023]
Abstract
BACKGROUND Collaborative learning is a group learning approach in which positive social interdependence within a group is key to better learning performance and future attitudes toward team practice. Recent attempts to replace a face-to-face environment with an online one have been developed using information communication technology. However, this raises the concern that online collaborative learning (OCL) may reduce positive social interdependence. Therefore, this study aimed to compare the degree of social interdependence in OCL with face-to-face environments and clarify aspects that affect social interdependence in OCL. METHODS We conducted a crossover study comparing online and face-to-face collaborative learning environments in a clinical reasoning class using team-based learning for medical students (n = 124) in 2021. The participants were randomly assigned to two cohorts: Cohort A began in an online environment, while Cohort B began in a face-to-face environment. At the study's midpoint, the two cohorts exchanged the environments as a washout. The participants completed surveys using the social interdependence in collaborative learning scale (SOCS) to measure their perceived positive social interdependence before and after the class. Changes in the mean SOCS scores were compared using paired t-tests. Qualitative data related to the characteristics of the online environment were obtained from the focus groups and coded using thematic analysis. RESULTS The matched-pair tests of SOCS showed significant progression between pre- and post-program scores in the online and face-to-face groups. There were no significant differences in overall SOCS scores between the two groups. Sub-analysis by subcategory showed significant improvement in boundary (discontinuities among individuals) and means interdependence (resources, roles, and tasks) in both groups, but outcome interdependence (goals and rewards) improved significantly only in the online group. Qualitative analysis revealed four major themes affecting social interdependence in OCL: communication, task-sharing process, perception of other groups, and working facilities. CONCLUSIONS There is a difference in the communication styles of students in face-to-face and online environments, and these various influences equalize the social interdependence in a face-to-face and online environment.
Collapse
Affiliation(s)
- Ikuo Shimizu
- Center for Medical Education and Clinical Training, Shinshu University, 3-1-1 Asahi, Matsumoto, 3908621 Japan
| | - Yasushi Matsuyama
- Medical Education Centre, Jichi Medical University, 3311-1 Yakushiji, Shimotsuke-shi, Tochigi Japan
| | - Robbert Duvivier
- Center for Educational Development and Research in Health Sciences (CEDAR), University Medical Center Groningen, Antonius Deusinglaan, 19713 AV Groningen, The Netherlands
| | - Cees van der Vleuten
- Department of Educational Development and Research, Faculty of Health, Medicine and Life Sciences, Maastricht University, Universiteitssingel 60, 6229 ER Maastricht, The Netherlands
| |
Collapse
|
5
|
Swanberg M, Woodson-Smith S, Pangaro L, Torre D, Maggio L. Factors and Interactions Influencing Direct Observation: A Literature Review Guided by Activity Theory. TEACHING AND LEARNING IN MEDICINE 2022; 34:155-166. [PMID: 34238091 DOI: 10.1080/10401334.2021.1931871] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/29/2020] [Revised: 04/19/2021] [Accepted: 05/11/2021] [Indexed: 06/13/2023]
Abstract
PhenomenonEnsuring that future physicians are competent to practice medicine is necessary for high quality patient care and safety. The shift toward competency-based education has placed renewed emphasis on direct observation via workplace-based assessments in authentic patient care contexts. Despite this interest and multiple studies focused on improving direct observation, challenges regarding the objectivity of this assessment approach remain underexplored and unresolved. Approach: We conducted a literature review of direct observation in authentic patient contexts by systematically searching databases PubMed, Embase, Web of Science, and ERIC. Included studies comprised original research conducted in the patient care context with authentic patients, either as a live encounter or a video recording of an actual encounter, which focused on factors affecting the direct observation of undergraduate medical education (UME) or graduate medical education (GME) trainees. Because the patient care context adds factors that contribute to the cognitive load of the learner and of the clinician-observer we focused our question on such contexts, which are most useful in judgments about advancement to the next level of training or practice. We excluded articles or published abstracts not conducted in the patient care context (e.g., OSCEs) or those involving simulation, allied health professionals, or non-UME/GME trainees. We also excluded studies focused on end-of-rotation evaluations and in-training evaluation reports. We extracted key data from the studies and used Activity Theory as a lens to identify factors affecting these observations and the interactions between them. Activity Theory provides a framework to understand and analyze complex human activities, the systems in which people work, and the interactions or tensions between multiple associated factors. Findings: Nineteen articles were included in the analysis; 13 involved GME learners and 6 UME learners. Of the 19, six studies were set in the operating room and four in the Emergency department. Using Activity Theory, we discovered that while numerous studies focus on rater and tool influences, very few study the impact of social elements. These are the rules that govern how the activity happens, the environment and members of the community involved in the activity and how completion of the activity is divided up among the members of the community. Insights: Viewing direct observation via workplace-based assessment through the lens of Activity Theory may enable educators to implement curricular changes to improve direct observation of assessment. Activity Theory may allow researchers to design studies to focus on the identified underexplored interactions and influences in relation to direct observation.
Collapse
Affiliation(s)
- Margaret Swanberg
- Department of Neurology, Uniformed Services University, Bethesda, Maryland, USA
| | - Sarah Woodson-Smith
- Department of Neurology, Naval Medical Center Portsmouth, Portsmouth, Virginia, USA
| | - Louis Pangaro
- Department of Medicine, Uniformed Services University, Bethesda, Maryland, USA
| | - Dario Torre
- Department of Medicine, Uniformed Services University, Bethesda, Maryland, USA
- Center for Health Professions Education, Uniformed Services University, Bethesda, Maryland, USA
| | - Lauren Maggio
- Department of Medicine, Uniformed Services University, Bethesda, Maryland, USA
- Center for Health Professions Education, Uniformed Services University, Bethesda, Maryland, USA
| |
Collapse
|
6
|
Ren DM, Abrams A, Banigan M, Batabyal R, Chamberlain JM, Garrow A, Izem R, Nicholson L, Ottolini M, Patterson M, Sarnacki R, Walsh HA, Zaveri P. Evaluation of Communication and Safety Behaviors During Hospital-Wide Code Response Simulation. Simul Healthc 2022; 17:e45-e50. [PMID: 33787552 DOI: 10.1097/sih.0000000000000575] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
Abstract
INTRODUCTION To understand the baseline quality of team communication behaviors at our organization, we implemented institution-wide simulation training and measured the performance of safety behaviors of ad hoc teams in emergent situations. METHODS Clinicians participated in 2 interprofessional video-recorded simulation scenarios, each followed by debriefing. Using a standardized evaluation instrument, 2 reviewers independently evaluated the presence or absence of desired team safety behaviors, including escalating care, sharing a mental model, establishing leadership, thinking out loud, and identifying roles and responsibilities. We also scored the quality of sharing the mental model, closed-loop communication, and overall team performance on a 7-point scale. Discordant reviews were resolved with scoring by an additional reviewer. RESULTS A total of 1404 clinicians participated in 398 simulation scenarios, resulting in 257 usable videos. Overall, teams exhibited desired behaviors at the following frequencies: escalating care, 85%; sharing mental models, 66%; verbally establishing leadership, 6%; thinking out loud, 87%; and identifying roles and responsibilities, 27%. Across all reviews, the quality of the graded behaviors (of 7 points) was 2.8 for shared mental models, 3.3 for closed-loop communication, and 3.2 for overall team performance. CONCLUSIONS In a simulation setting with ad hoc teams, there was variable performance on completing safety behaviors and only a fair quality of graded communication behaviors. These results establish a baseline assessment of communication and teamwork behaviors and will guide future quality improvement interventions.
Collapse
Affiliation(s)
- Dennis M Ren
- From the Division of Emergency Medicine (D.M.R., R.B., J.M.C, P.Z.), Children's National Hospital, Washington, DC; Division of Emergency Medicine (A.A.), Children's Hospital Colorado, Aurora, CO; Department of Anesthesiology and Critical Care Medicine (M.B.), Children's Hospital of Philadelphia, Philadelphia, PA; Department of Pediatrics (R.B., J.M.C., A.G., R.I., P.Z.), George Washington University School of Medicine and Health Sciences; Simulation Program (L.N., H.A.W.), Children's National Hospital; School of Nursing & Allied Health (A.G.), Liverpool John Moores University in Liverpool, England; Division of Biostatistics and Study Methodology (R.I.), Children's Research Institute, Children's National Hospital, Washington, DC; Department of Pediatrics (M.O.), Maine Medical Center, Portland, ME; Department of Emergency Medicine (M.P.), University of Florida College of Medicine; University of Florida Center for Experiential Learning and Simulation (M.P.), University of Florida College of Medicine, Gainesville, FL; and Global Health Initiative Children's National Hospital (R.S.), Washington, DC
| | | | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|
7
|
Scheurer JM, Davey C, Pereira AG, Olson APJ. Building a Shared Mental Model of Competence Across the Continuum: Trainee Perceptions of Subinternships for Residency Preparation. JOURNAL OF MEDICAL EDUCATION AND CURRICULAR DEVELOPMENT 2021; 8:23821205211063350. [PMID: 34988291 PMCID: PMC8721691 DOI: 10.1177/23821205211063350] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/04/2021] [Revised: 11/09/2021] [Indexed: 06/01/2023]
Abstract
INTRODUCTION Toward a vision of competency-based medical education (CBME) spanning the undergraduate to graduate medical education (GME) continuum, University of Minnesota Medical School (UMMS) developed the Subinternship in Critical Care (SICC) offered across specialties and sites. Explicit course objectives and assessments focus on internship preparedness, emphasizing direct observation of handovers (Core Entrustable Professional Activity, "EPA," 8) and cross-cover duties (EPA 10). METHODS To evaluate students' perceptions of the SICC's and other clerkships' effectiveness toward internship preparedness, all 2016 and 2017 UMMS graduates in GME training (n = 440) were surveyed regarding skill development and assessment among Core EPAs 1, 4, 6, 8, 9, 10. Analysis included descriptive statistics plus chi-squared and Kappa agreement tests. RESULTS Respondents (n = 147, response rate 33%) rated the SICC as a rotation during which they gained most competence among EPAs both more (#4, 57% rated important; #8, 75%; #10, 70%) and less explicit (#6, 53%; #9, 69%) per rotation objectives. Assessments of EPA 8 (80% rated important) and 10 (76%) were frequently perceived as important toward residency preparedness. Agreement between importance of EPA development and assessment was moderate (Kappa = 0.40-0.59, all surveyed EPAs). CONCLUSIONS Graduates' perceptions support the SICC's educational utility and assessments. Based on this and other insight from the SICC, the authors propose implications toward collectively envisioning the continuum of physician competency.
Collapse
Affiliation(s)
- Johannah M. Scheurer
- Department of Pediatrics, University of Minnesota Medical School, Minneapolis, MN, USA
| | - Cynthia Davey
- Clinical and Translational Science Institute, University of Minnesota, Minneapolis, MN, USA
| | - Anne G. Pereira
- University of Minnesota Medical School, Minneapolis, MN, USA
| | - Andrew P. J. Olson
- Department of Pediatrics, University of Minnesota Medical School, Minneapolis, MN, USA
- Department of Medicine, University of Minnesota Medical School, Minneapolis, MN, USA
| |
Collapse
|
8
|
Young JQ, Sugarman R, Schwartz J, O'Sullivan PS. Faculty and Resident Engagement With a Workplace-Based Assessment Tool: Use of Implementation Science to Explore Enablers and Barriers. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2020; 95:1937-1944. [PMID: 32568853 DOI: 10.1097/acm.0000000000003543] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
PURPOSE Implementation of workplace-based assessment programs has encountered significant challenges. Faculty and residents alike often have a negative view of these programs as "tick-box" or "jump through the hoops" exercises. A number of recommendations have been made to address these challenges. To understand the experience with a workplace-based assessment tool that follows many of these recommendations, the authors conducted a qualitative study using the Consolidated Framework for Implementation Research (CFIR) to identify enablers and barriers to engagement with the tool. METHOD The Psychopharmacotherapy-Structured Clinical Observation (P-SCO) is a direct observation tool designed to assess resident performance during a psychiatric medication management visit. From August 2017 to February 2018, the P-SCO was implemented in the outpatient continuity clinics for second- and third-year residents at Zucker Hillside Hospital/Northwell Health. In February and March 2019, the authors conducted semistructured interviews of participating faculty and residents. Interview guides based on the CFIR were used to capture the enablers and barriers to engagement. Interview transcripts were independently coded. Codes were then organized into themes relevant to the domains of the CFIR. RESULTS Ten faculty and 10 residents were interviewed. Overall, participants had a positive experience with the P-SCO. Enabling factors for faculty and residents included the ongoing training, design features of the P-SCO, predisposing beliefs, dedicated faculty time, and the perception that the P-SCO improved verbal feedback quality. Barriers for faculty included checklist length and discomfort with feedback that threatens identity, and barriers for residents included faculty variability in timeliness and quality of feedback and minimal review of the feedback after initial receipt. CONCLUSIONS This study demonstrates that the negative experience of faculty and residents with workplace-based assessment tools shown in prior studies can be overcome, at least in part, when specific implementation strategies are pursued. The findings provide guidance for future research and implementation efforts.
Collapse
Affiliation(s)
- John Q Young
- J.Q. Young is professor of psychiatry and vice chair for education, Department of Psychiatry, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, New York, and Zucker Hillside Hospital at Northwell Health, Glen Oaks, New York
| | - Rebekah Sugarman
- R. Sugarman is a research assistant, Department of Psychiatry, Zucker Hillside Hospital at Northwell Health, Glen Oaks, New York
| | - Jessica Schwartz
- J. Schwartz is a resident, Department of Psychiatry, Zucker Hillside Hospital at Northwell Health, Glen Oaks, New York
| | - Patricia S O'Sullivan
- P.S. O'Sullivan is professor, Department of Medicine, and director of research and development in medical education, University of California, San Francisco, School of Medicine, San Francisco, California
| |
Collapse
|
9
|
Lestari E, Scherpbier A, Stalmeijer R. Stimulating Students' Interprofessional Teamwork Skills Through Community-Based Education: A Mixed Methods Evaluation. J Multidiscip Healthc 2020; 13:1143-1155. [PMID: 33116560 PMCID: PMC7568678 DOI: 10.2147/jmdh.s267732] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2020] [Accepted: 09/24/2020] [Indexed: 12/31/2022] Open
Abstract
Background Interprofessional education (IPE) is suggested as a good means to prepare future healthcare professionals for collaborative work in interprofessional teams enabling them to solve complex health problems. Previous studies have advocated experiential IPE, including community-based IPE (CBIPE). This study aims to evaluate a CBIPE programme by exploring the students’ perception toward CBIPE design and toward groups’ teamwork. Methods To identify students’ perceptions of teamwork, the Interprofessional Teamwork Evaluation questionnaire was administered to 254 students of medical, nursing and midwifery programme. Three uni-professional focus group (FG) discussions were conducted to analyse the students’ perception of the design of community-based education and underlying reasons for teamwork. Results FGs reported three aspects that influence skills development in collaborative practice among students that shed light on why midwifery and nursing students held less positive perceptions of communication and mutual support: 1) communication gap due to lack of confidence, 2) contrasting ways of thinking affect communication in decision-making, and 3) the leadership culture in the health services. Conclusion A CBIPE programme was successfully implemented at Universitas Islam Sultan Agung. It demonstrated that students in the health professions can develop skills in collaborative practice despite having some problems with communication and mutual support.
Collapse
Affiliation(s)
- Endang Lestari
- Medical and Health Professions Education Unit, Faculty of Medicine, Universitas Islam Sultan Agung, Semarang, Indonesia
| | - Albert Scherpbier
- School of Health Professions Education, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, the Netherlands
| | - Renee Stalmeijer
- School of Health Professions Education, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, the Netherlands
| |
Collapse
|
10
|
Young JQ, Sugarman R, Schwartz J, McClure M, O'Sullivan PS. A mobile app to capture EPA assessment data: Utilizing the consolidated framework for implementation research to identify enablers and barriers to engagement. PERSPECTIVES ON MEDICAL EDUCATION 2020; 9:210-219. [PMID: 32504446 PMCID: PMC7459074 DOI: 10.1007/s40037-020-00587-z] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/08/2023]
Abstract
INTRODUCTION Mobile apps that utilize the framework of entrustable professional activities (EPAs) to capture and deliver feedback are being implemented. If EPA apps are to be successfully incorporated into programmatic assessment, a better understanding of how they are experienced by the end-users will be necessary. The authors conducted a qualitative study using the Consolidated Framework for Implementation Research (CFIR) to identify enablers and barriers to engagement with an EPA app. METHODS Structured interviews of faculty and residents were conducted with an interview guide based on the CFIR. Transcripts were independently coded by two study authors using directed content analysis. Differences were resolved via consensus. The study team then organized codes into themes relevant to the domains of the CFIR. RESULTS Eight faculty and 10 residents chose to participate in the study. Both faculty and residents found the app easy to use and effective in facilitating feedback immediately after the observed patient encounter. Faculty appreciated how the EPA app forced brief, distilled feedback. Both faculty and residents expressed positive attitudes and perceived the app as aligned with the department's philosophy. Barriers to engagement included faculty not understanding the EPA framework and scale, competing clinical demands, residents preferring more detailed feedback and both faculty and residents noting that the app's feedback should be complemented by a tool that generates more systematic, nuanced, and comprehensive feedback. Residents rarely if ever returned to the feedback after initial receipt. DISCUSSION This study identified key enablers and barriers to engagement with the EPA app. The findings provide guidance for future research and implementation efforts focused on the use of mobile platforms to capture direct observation feedback.
Collapse
Affiliation(s)
- John Q Young
- Donald and Barbara Zucker School of Medicine at Hofstra/Northwell and the Zucker Hillside Hospital at Northwell Health, Hempstead, NY, USA.
| | - Rebekah Sugarman
- Donald and Barbara Zucker School of Medicine at Hofstra/Northwell and the Zucker Hillside Hospital at Northwell Health, Hempstead, NY, USA
| | - Jessica Schwartz
- Donald and Barbara Zucker School of Medicine at Hofstra/Northwell and the Zucker Hillside Hospital at Northwell Health, Hempstead, NY, USA
| | - Matthew McClure
- Donald and Barbara Zucker School of Medicine at Hofstra/Northwell and the Zucker Hillside Hospital at Northwell Health, Hempstead, NY, USA
| | - Patricia S O'Sullivan
- Department of Medicine, University of California at San Francisco School of Medicine, San Francisco, USA
| |
Collapse
|
11
|
Young JQ. Advancing Our Understanding of Narrative Comments Generated by Direct Observation Tools: Lessons From the Psychopharmacotherapy-Structured Clinical Observation. J Grad Med Educ 2019; 11:570-579. [PMID: 31636828 PMCID: PMC6795331 DOI: 10.4300/jgme-d-19-00207.1] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/23/2019] [Revised: 07/07/2019] [Accepted: 08/05/2019] [Indexed: 11/06/2022] Open
Abstract
BACKGROUND While prior research has focused on the validity of quantitative ratings generated by direct observation tools, much less is known about the written comments. OBJECTIVE This study examines the quality of written comments and their relationship with checklist scores generated by a direct observation tool, the Psychopharmacotherapy-Structured Clinical Observation (P-SCO). METHODS From 2008 to 2012, faculty in a postgraduate year 3 psychiatry outpatient clinic completed 601 P-SCOs. Twenty-five percent were randomly selected from each year; the sample included 8 faculty and 57 residents. To assess quality, comments were coded for valence (reinforcing or corrective), behavioral specificity, and content. To assess the relationship between comments and scores, the authors calculated the correlation between comment and checklist score valence and examined the degree to which comments and checklist scores addressed the same content. RESULTS Ninety-one percent of the comments were behaviorally specific. Sixty percent were reinforcing, and 40% were corrective. Eight themes were identified, including 2 constructs not adequately represented by the checklist. Comment and checklist score valence was moderately correlated (Spearman's rho = 0.57, P < .001). Sixty-seven percent of high and low checklist scores were associated with a comment of the same valence and content. Only 50% of overall comments were associated with a checklist score of the same valence and content. CONCLUSIONS A direct observation tool such as the P-SCO can generate high-quality written comments. Narrative comments both explain checklist scores and convey unique content. Thematic coding of comments can improve the content validity of a checklist.
Collapse
|
12
|
Higham H, Greig PR, Rutherford J, Vincent L, Young D, Vincent C. Observer-based tools for non-technical skills assessment in simulated and real clinical environments in healthcare: a systematic review. BMJ Qual Saf 2019; 28:672-686. [DOI: 10.1136/bmjqs-2018-008565] [Citation(s) in RCA: 27] [Impact Index Per Article: 5.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2018] [Revised: 04/17/2019] [Accepted: 04/23/2019] [Indexed: 12/18/2022]
Abstract
BackgroundOver the past three decades multiple tools have been developed for the assessment of non-technical skills (NTS) in healthcare. This study was designed primarily to analyse how they have been designed and tested but also to consider guidance on how to select them.ObjectivesTo analyse the context of use, method of development, evidence of validity (including reliability) and usability of tools for the observer-based assessment of NTS in healthcare.DesignSystematic review.Data sourcesSearch of electronic resources, including PubMed, Embase, CINAHL, ERIC, PsycNet, Scopus, Google Scholar and Web of Science. Additional records identified through searching grey literature (OpenGrey, ProQuest, AHRQ, King’s Fund, Health Foundation).Study selectionStudies of observer-based tools for NTS assessment in healthcare professionals (or undergraduates) were included if they: were available in English; published between January 1990 and March 2018; assessed two or more NTS; were designed for simulated or real clinical settings and had provided evidence of validity plus or minus usability. 11,101 articles were identified. After limits were applied, 576 were retrieved for evaluation and 118 articles included in this review.ResultsOne hundred and eighteen studies describing 76 tools for assessment of NTS in healthcare met the eligibility criteria. There was substantial variation in the method of design of the tools and the extent of validity, and usability testing. There was considerable overlap in the skills assessed, and the contexts of use of the tools.ConclusionThis study suggests a need for rationalisation and standardisation of the way we assess NTS in healthcare and greater consistency in how tools are developed and deployed.
Collapse
|
13
|
Bismilla Z, Boyle T, Mangold K, Van Ittersum W, White ML, Zaveri P, Mallory L. Development of a Simulation-Based Interprofessional Teamwork Assessment Tool. J Grad Med Educ 2019; 11:168-176. [PMID: 31024648 PMCID: PMC6476092 DOI: 10.4300/jgme-d-18-00729.1] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/19/2018] [Revised: 01/22/2019] [Accepted: 02/11/2019] [Indexed: 12/29/2022] Open
Abstract
BACKGROUND The Accreditation Council for Graduate Medical Education (ACGME) Milestone projects required each specialty to identify essential skills and develop means of assessment with supporting validity evidence for trainees. Several specialties rate trainees on a milestone subcompetency related to working in interprofessional teams. A tool to assess trainee competence in any role on an interprofessional team in a variety of scenarios would be valuable and suitable for simulation-based assessment. OBJECTIVE We developed a tool for simulation settings that assesses interprofessional teamwork in trainees. METHODS In 2015, existing tools that assess teamwork or interprofessionalism using direct observation were systematically reviewed for appropriateness, generalizability, adaptability, ease of use, and resources required. Items from these tools were included in a Delphi method with multidisciplinary pediatrics experts using an iterative process from June 2016 to January 2017 to develop an assessment tool. RESULTS Thirty-one unique tools were identified. A 2-stage review narrowed this list to 5 tools, and 81 items were extracted. Twenty-two pediatrics experts participated in 4 rounds of Delphi surveys, with response rates ranging from 82% to 100%. Sixteen items reached consensus for inclusion in the final tool. A global 4-point rating scale from novice to proficient was developed. CONCLUSIONS A novel tool to assess interprofessional teamwork for individual trainees in a simulated setting was developed using a systematic review and Delphi methodology. This is the first step to establish the validity evidence necessary to use this tool for competency-based assessment.
Collapse
|
14
|
Delaney KR, Karnik NS. Building a Child Mental Health Workforce for the 21st Century: Closing the Training Gap. J Prof Nurs 2019; 35:133-137. [PMID: 30902405 DOI: 10.1016/j.profnurs.2018.07.002] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2017] [Revised: 06/15/2018] [Accepted: 07/03/2018] [Indexed: 11/17/2022]
Affiliation(s)
- Kathleen R Delaney
- Psychiatric Mental Health Nurse Practitioner Program, Rush College of Nursing, Department of Community, Systems and Mental Health, 600 S. Paulina St, Chicago, IL 60612, United States of America.
| | - Niranjan S Karnik
- Department of Psychiatry, Section on Population Behavioral Health, Department of Psychiatry, Rush University Medical Center, 1645 West Jackson Blvd., Suite 302, Chicago, IL 60612, United States of America.
| |
Collapse
|
15
|
Duijn CCMA, Dijk EJV, Mandoki M, Bok HGJ, Cate OTJT. Assessment Tools for Feedback and Entrustment Decisions in the Clinical Workplace: A Systematic Review. JOURNAL OF VETERINARY MEDICAL EDUCATION 2019; 46:340-352. [PMID: 31460844 DOI: 10.3138/jvme.0917-123r] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
BACKGROUND: Entrustable Professional Activities (EPAs) combine feedback and evaluation with a permission to act under a specified level of supervision and the possibility to schedule learners for clinical service. This literature review aims to identify workplace-based assessment tools that indicate progression toward unsupervised practice, suitable for entrustment decisions and feedback to learners. METHODS: A systematic search was performed in the PubMed, Embase, ERIC, and PsycINFO databases. Based on title/abstract and full text, articles were selected using predetermined inclusion and exclusion criteria. Information on workplace-based assessment tools was extracted using data coding sheets. The methodological quality of studies was assessed using the medical education research study quality instrument (MERSQI). RESULTS: The search yielded 6,371 articles (180 were evaluated in full text). In total, 80 articles were included, identifying 67 assessment tools. Only a few studies explicitly mentioned assessment tools used as a resource for entrustment decisions. Validity evidence was frequently reported, and the MERSQI score was 10.0 on average. CONCLUSIONS: Many workplace-based assessment tools were identified that potentially support learners with feedback on their development and support supervisors with providing feedback. As expected, only few articles referred to entrustment decisions. Nevertheless, the existing tools or the principals could be used for entrustment decisions, supervision level, or autonomy.
Collapse
|
16
|
Young JQ, Rasul R, O'Sullivan PS. Evidence for the Validity of the Psychopharmacotherapy-Structured Clinical Observation Tool: Results of a Factor and Time Series Analysis. ACADEMIC PSYCHIATRY : THE JOURNAL OF THE AMERICAN ASSOCIATION OF DIRECTORS OF PSYCHIATRIC RESIDENCY TRAINING AND THE ASSOCIATION FOR ACADEMIC PSYCHIATRY 2018; 42:759-764. [PMID: 29951950 DOI: 10.1007/s40596-018-0928-0] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/28/2017] [Accepted: 04/18/2018] [Indexed: 06/08/2023]
Abstract
OBJECTIVE The Psychopharmacotherapy-Structured Clinical Observation (P-SCO) is a direct observation tool designed to assess resident performance of a medication visit. This study examines two dimensions of validity for the P-SCO: internal structure and how scores correlate with another variable associated with competence (experience). METHODS The faculty completed 601 P-SCOs over 4 years. Multilevel exploratory factor analysis was performed with minimum thresholds for eigenvalue (≥ 1.0) and proportion of variance explained (≥ 5.0%). Internal reliability was assessed with Cronbach alpha. To examine how scores changed with experience, mean ratings (1-4 scale) were calculated for each factor by quarter of the academic year. Separate linear mixed models were also performed. RESULTS The analysis yielded three factors that explained 50% of the variance and demonstrated high internal reliability: affective tasks (alpha = 0.90), cognitive tasks (alpha = 0.84), and hard tasks (alpha = 0.74). Items within "hard tasks" were assessment of substance use, violence risk, and adherence, and inquiry about interactions with other providers. Monitoring adverse effects did not load on the hard task factor but also had overall low mean ratings. Compared to the first quarter, fourth quarter scores for affective tasks (b = 0.54, p < 0.01) and hard tasks (b = 0.46, p = 0.02) were significantly improved while cognitive tasks had a non-significant increase. For the hard tasks, the proportion of residents with a low mean rating improved but was still over 30% during the fourth quarter. CONCLUSIONS The results provide evidence for the validity of the P-SCO with respect to its internal structure and how scores correlate with experience. Curricular implications are explored, especially for the tasks that were hard to learn.
Collapse
Affiliation(s)
- John Q Young
- Zucker School of Medicine at Hofstra/Northwell, Hempstead, NY, USA.
| | | | | |
Collapse
|
17
|
Sebok‐Syer SS, Chahine S, Watling CJ, Goldszmidt M, Cristancho S, Lingard L. Considering the interdependence of clinical performance: implications for assessment and entrustment. MEDICAL EDUCATION 2018; 52:970-980. [PMID: 29676054 PMCID: PMC6120474 DOI: 10.1111/medu.13588] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/04/2017] [Revised: 12/14/2017] [Accepted: 02/20/2018] [Indexed: 05/05/2023]
Abstract
INTRODUCTION Our ability to assess independent trainee performance is a key element of competency-based medical education (CBME). In workplace-based clinical settings, however, the performance of a trainee can be deeply entangled with others on the team. This presents a fundamental challenge, given the need to assess and entrust trainees based on the evolution of their independent clinical performance. The purpose of this study, therefore, was to understand what faculty members and senior postgraduate trainees believe constitutes independent performance in a variety of clinical specialty contexts. METHODS Following constructivist grounded theory, and using both purposive and theoretical sampling, we conducted individual interviews with 11 clinical teaching faculty members and 10 senior trainees (postgraduate year 4/5) across 12 postgraduate specialties. Constant comparative inductive analysis was conducted. Return of findings was also carried out using one-to-one sessions with key informants and public presentations. RESULTS Although some independent performances were described, participants spoke mostly about the exceptions to and disclaimers about these, elaborating their sense of the interdependence of trainee performances. Our analysis of these interdependence patterns identified multiple configurations of coupling, with the dominant being coupling of trainee and supervisor performance. We consider how the concept of coupling could advance workplace-based assessment efforts by supporting models that account for the collective dimensions of clinical performance. CONCLUSION These findings call into question the assumption of independent performance, and offer an important step toward measuring coupled performance. An understanding of coupling can help both to better distinguish independent and interdependent performances, and to consider revising workplace-based assessment approaches for CBME.
Collapse
Affiliation(s)
- Stefanie S Sebok‐Syer
- Centre for Education Research and InnovationSchulich School of Medicine and DentistryWestern UniversityLondonOntarioCanada
| | - Saad Chahine
- Centre for Education Research and InnovationSchulich School of Medicine and DentistryWestern UniversityLondonOntarioCanada
| | - Christopher J Watling
- Centre for Education Research and InnovationSchulich School of Medicine and DentistryWestern UniversityLondonOntarioCanada
| | - Mark Goldszmidt
- Centre for Education Research and InnovationSchulich School of Medicine and DentistryWestern UniversityLondonOntarioCanada
| | - Sayra Cristancho
- Centre for Education Research and InnovationSchulich School of Medicine and DentistryWestern UniversityLondonOntarioCanada
| | - Lorelei Lingard
- Centre for Education Research and InnovationSchulich School of Medicine and DentistryWestern UniversityLondonOntarioCanada
| |
Collapse
|
18
|
Andrews JS, Bale JF, Soep JB, Long M, Carraccio C, Englander R, Powell D. Education in Pediatrics Across the Continuum (EPAC): First Steps Toward Realizing the Dream of Competency-Based Education. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2018; 93:414-420. [PMID: 29023245 DOI: 10.1097/acm.0000000000002020] [Citation(s) in RCA: 61] [Impact Index Per Article: 10.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/13/2023]
Abstract
The Education in Pediatrics Across the Continuum (EPAC) Study Group is developing the first competency-based, time-variable progression from undergraduate medical education (UME) to graduate medical education (GME) in the history of medical education in the United States. EPAC, an innovation project sponsored by the Association of American Medical Colleges and supported by the Josiah Macy Jr. Foundation, was developed through a collaboration between five medical schools and multiple professional organizations with an interest in undergraduate and graduate medical education. The planning and implementation process demanded cooperatively addressing practical barriers such as education requirements for licensure and developing approaches to learner assessment that provided meaningful information about competency. Each participating school now has at least three cohorts of learners participating, and the program is transitioning its first cohort of students from UME to GME based on achievement of predetermined competencies that allow this transition. Members of the first cohort of learners in this program have begun their pediatric residency training at different times beginning in late 2016, confirming the feasibility of competency-based advancement from UME to GME in pediatrics. Although there is still much to learn about the outcomes of EPAC learners' professional development in residency training and beyond, EPAC has defined an operational approach to a different path through medical school and into residency training, based on the attainment of competence.
Collapse
Affiliation(s)
- John S Andrews
- J.S. Andrews is vice chair for education and associate professor, Department of Pediatrics, and associate dean for graduate medical education, University of Minnesota Medical School, Minneapolis, Minnesota; ORCID: http://orcid.org/0000-0002-2008-2686. J.F. Bale is professor of pediatrics and neurology, Department of Pediatrics, University of Utah School of Medicine, Salt Lake City, Utah. J.B. Soep is associate professor of pediatrics, Department of Pediatrics, and pediatric clerkship director, University of Colorado School of Medicine, Aurora, Colorado. M. Long is associate professor of pediatrics, Department of Pediatrics, University of California, San Francisco, School of Medicine, San Francisco, California; ORCID: http://orcid.org/0000-0002-8399-5589. C. Carraccio is vice president for competency-based assessment programs, American Board of Pediatrics, Chapel Hill, North Carolina. R. Englander is professor of pediatrics, Department of Pediatrics, and associate dean for undergraduate medical education, University of Minnesota Medical School, Minneapolis, Minnesota. D. Powell is professor of laboratory medicine and pathology, Department of Laboratory Medicine and Pathology, and dean emerita, University of Minnesota Medical School, Minneapolis, Minnesota
| | | | | | | | | | | | | |
Collapse
|
19
|
Shrader S, Farland MZ, Danielson J, Sicat B, Umland EM. A Systematic Review of Assessment Tools Measuring Interprofessional Education Outcomes Relevant to Pharmacy Education. AMERICAN JOURNAL OF PHARMACEUTICAL EDUCATION 2017; 81:119. [PMID: 28970620 PMCID: PMC5607729 DOI: 10.5688/ajpe816119] [Citation(s) in RCA: 54] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/02/2016] [Accepted: 08/03/2016] [Indexed: 05/19/2023]
Abstract
Objective. To identify and describe the available quantitative tools that assess interprofessional education (IPE) relevant to pharmacy education. Methods. A systematic approach was used to identify quantitative IPE assessment tools relevant to pharmacy education. The search strategy included the National Center for Interprofessional Practice and Education Resource Exchange (Nexus) website, a systematic search of the literature, and a manual search of journals deemed likely to include relevant tools. Results. The search identified a total of 44 tools from the Nexus website, 158 abstracts from the systematic literature search, and 570 abstracts from the manual search. A total of 36 assessment tools met the criteria to be included in the summary, and their application to IPE relevant to pharmacy education was discussed. Conclusion. Each of the tools has advantages and disadvantages. No single comprehensive tool exists to fulfill assessment needs. However, numerous tools are available that can be mapped to IPE-related accreditation standards for pharmacy education.
Collapse
Affiliation(s)
- Sarah Shrader
- University of Kansas School of Pharmacy, Lawrence, Kansas
| | | | | | | | | |
Collapse
|
20
|
Gonzalo JD, Baxley E, Borkan J, Dekhtyar M, Hawkins R, Lawson L, Starr SR, Skochelak S. Priority Areas and Potential Solutions for Successful Integration and Sustainment of Health Systems Science in Undergraduate Medical Education. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2017; 92:63-69. [PMID: 27254015 DOI: 10.1097/acm.0000000000001249] [Citation(s) in RCA: 18] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/05/2023]
Abstract
Educators, policy makers, and health systems leaders are calling for significant reform of undergraduate medical education (UME) and graduate medical education (GME) programs to meet the evolving needs of the health care system. Nationally, several schools have initiated innovative curricula in both classroom and workplace learning experiences to promote education in health systems science (HSS), which includes topics such as value-based care, health system improvement, and population and public health. However, the successful implementation of HSS curricula across schools is challenged by issues of curriculum design, assessment, culture, and accreditation, among others. In this report of a working conference using thematic analysis of workshop recommendations and experiences from 11 U.S. medical schools, the authors describe seven priority areas for the successful integration and sustainment of HSS in educational programs, and associated challenges and potential solutions. In 2015, following regular HSS workgroup phone calls and an Accelerating Change in Medical Education consortium-wide meeting, the authors identified the priority areas: partner with licensing, certifying, and accrediting bodies; develop comprehensive, standardized, and integrated curricula; develop, standardize, and align assessments; improve the UME to GME transition; enhance teachers' knowledge and skills, and incentives for teachers; demonstrate value added to the health system; and address the hidden curriculum. These priority areas and their potential solutions can be used by individual schools and HSS education collaboratives to further outline and delineate the steps needed to create, deliver, study, and sustain effective HSS curricula with an eye toward integration with the basic and clinical sciences curricula.
Collapse
Affiliation(s)
- Jed D Gonzalo
- J.D. Gonzalo is assistant professor of medicine and public health sciences and associate dean for health systems education, Penn State College of Medicine, Hershey, Pennsylvania. E. Baxley is senior associate dean of academic affairs, Brody School of Medicine at East Carolina University, Greenville, North Carolina. J. Borkan is chair and professor of family medicine and assistant dean for primary care-population health program planning, Alpert Medical School of Brown University, Providence, Rhode Island. M. Dekhtyar is research associate, Medical Education Outcomes, American Medical Association, Chicago, Illinois. R. Hawkins is vice president, Medical Education Outcomes, American Medical Association, Chicago, Illinois. L. Lawson is assistant dean for curriculum, assessment, and clinical academic affairs and assistant professor of emergency medicine, Brody School of Medicine at East Carolina University, Greenville, North Carolina. S.R. Starr is assistant professor of pediatric and adolescent medicine and director of science of health care delivery education, Mayo Medical School, Mayo Clinic College of Medicine, Rochester, Minnesota. S. Skochelak is group vice president of medical education, American Medical Association, Chicago, Illinois
| | | | | | | | | | | | | | | |
Collapse
|
21
|
Lie DA, Richter-Lagha R, Forest CP, Walsh A, Lohenry K. When less is more: validating a brief scale to rate interprofessional team competencies. MEDICAL EDUCATION ONLINE 2017; 22:1314751. [PMID: 28475438 PMCID: PMC5508637 DOI: 10.1080/10872981.2017.1314751] [Citation(s) in RCA: 25] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/22/2016] [Accepted: 03/29/2017] [Indexed: 05/22/2023]
Abstract
BACKGROUND There is a need for validated and easy-to-apply behavior-based tools for assessing interprofessional team competencies in clinical settings. The seven-item observer-based Modified McMaster-Ottawa scale was developed for the Team Objective Structured Clinical Encounter (TOSCE) to assess individual and team performance in interprofessional patient encounters. OBJECTIVE We aimed to improve scale usability for clinical settings by reducing item numbers while maintaining generalizability; and to explore the minimum number of observed cases required to achieve modest generalizability for giving feedback. DESIGN We administered a two-station TOSCE in April 2016 to 63 students split into 16 newly-formed teams, each consisting of four professions. The stations were of similar difficulty. We trained sixteen faculty to rate two teams each. We examined individual and team performance scores using generalizability (G) theory and principal component analysis (PCA). RESULTS The seven-item scale shows modest generalizability (.75) with individual scores. PCA revealed multicollinearity and singularity among scale items and we identified three potential items for removal. Reducing items for individual scores from seven to four (measuring Collaboration, Roles, Patient/Family-centeredness, and Conflict Management) changed scale generalizability from .75 to .73. Performance assessment with two cases is associated with reasonable generalizability (.73). Students in newly-formed interprofessional teams show a learning curve after one patient encounter. Team scores from a two-station TOSCE demonstrate low generalizability whether the scale consisted of four (.53) or seven items (.55). CONCLUSION The four-item Modified McMaster-Ottawa scale for assessing individual performance in interprofessional teams retains the generalizability and validity of the seven-item scale. Observation of students in teams interacting with two different patients provides reasonably reliable ratings for giving feedback. The four-item scale has potential for assessing individual student skills and the impact of IPE curricula in clinical practice settings. ABBREVIATIONS IPE: Interprofessional education; SP: Standardized patient; TOSCE: Team objective structured clinical encounter.
Collapse
Affiliation(s)
- Désirée A. Lie
- Department of Family Medicine, Keck School of Medicine of the University of Southern California, CA, USA
| | - Regina Richter-Lagha
- Department of Family Medicine, Keck School of Medicine of the University of Southern California, CA, USA
| | - Christopher P. Forest
- Department of Family Medicine, Keck School of Medicine of the University of Southern California, CA, USA
| | - Anne Walsh
- Department of Family Medicine, Keck School of Medicine of the University of Southern California, CA, USA
| | - Kevin Lohenry
- Department of Family Medicine, Keck School of Medicine of the University of Southern California, CA, USA
| |
Collapse
|
22
|
Higgins Joyce A. Team-Based Simulation for Medical Student Handoff Education. MEDEDPORTAL : THE JOURNAL OF TEACHING AND LEARNING RESOURCES 2016; 12:10486. [PMID: 30984828 PMCID: PMC6440419 DOI: 10.15766/mep_2374-8265.10486] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/04/2015] [Accepted: 10/03/2016] [Indexed: 06/09/2023]
Abstract
INTRODUCTION Good handoffs require teamwork, clear communication, a cognitively safe environment, and a good understanding of the patient's medical needs. Complex tertiary care training institutions require multiple handoffs over a patient's time in the hospital. Medical students need better handoff education. We designed a handoff training exercise using a simulated patient care environment for students to practice and observe multiple handoffs over time. METHODS An initial large-group didactic session provides direct instruction on handoffs. In small groups, students are subsequently assigned roles as individual physicians in a chain of providers during a simulated patient hospitalization over several days, with an additional student as an observer. Blinded to any prior discussion, student physicians sequentially give and receive handoffs about the patient from a previous physician as their simulated hospital course evolves. The observer shares his or her insights, and a large-group structured debriefing exercise follows. RESULTS In both 2015 and 2016, we implemented this session with a cohort of 30 fourth-year medical students. Most recently we implemented this with chief residents, and a group of 20 third-year pediatric clerkship students. We reviewed a selection of the discussion guides and found reporters/ observers noted that participants stated the primary problem 95% of the time (19 of 20 handoffs), the patient acuity 90% of the time (18 of 20 handoffs), and a clear contingency plan 85% of the time (17 of 20 handoffs). DISCUSSION We found students initially to be preoccupied with making correct clinical decisions instead of giving effective handoffs, and consequently we clarified details and adjusted clinical information to be more transparent. Results suggest the session achieves our goals of using a structured handoff method to communicate higher-level information and debrief effectively with peers. We envision this activity to be applicable to teaching handoffs to residents, nurses, and other health care professionals, as well as possibly in diverse clinical environments or in collaboration with outpatient providers.
Collapse
Affiliation(s)
- Alanna Higgins Joyce
- Assistant Professor, Department of Pediatrics, Northwestern University The Feinberg School of Medicine
| |
Collapse
|
23
|
Roy B, Chheda SG, Bates C, Dunn K, Karani R, Willett LL. For the General Internist: A Summary of Key Innovations in Medical Education. J Gen Intern Med 2016; 31:941-6. [PMID: 27084757 PMCID: PMC4945558 DOI: 10.1007/s11606-016-3669-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/22/2015] [Revised: 02/19/2016] [Accepted: 03/08/2016] [Indexed: 10/21/2022]
Abstract
We conducted a review of published medical education articles to identify high-quality research and innovation relevant to educators in general medicine. Our review team consisted of six general internists with expertise in medical education and a professional medical librarian. We manually searched 15 journals in pairs (a total of 3062 citations) for original research articles in medical education published in 2014. Each pair of reviewers independently rated the relevance, importance, and generalizability of articles on medical education in their assigned journals using a 27-point scale (maximum of 9 points for each characteristic). From this list, each team member independently reviewed the 22 articles that received a score of 20 or higher from both initial reviewers, and for each selected article rated the quality and global relevance for the generalist educator. We included the seven top-rated articles for presentation in this review, and categorized the studies into four general themes: continuity clinic scheduling, remediation, interprofessional education, and quality improvement and patient safety. We summarized key findings and identified significant limitations of each study. Further studies assessing patient outcomes are needed to strengthen the literature in medical education. This summary of relevant medical education articles can inform future research, teaching, and practice.
Collapse
Affiliation(s)
- Brita Roy
- Section of General Internal Medicine, Yale University School of Medicine, PO Box 208025, New Haven, CT, 06520-8025, USA.
| | - Shobhina G Chheda
- University of Wisconsin School of Medicine and Public Health, Madison, WI, USA
| | - Carol Bates
- Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
| | - Kathel Dunn
- National Library of Medicine, Bethesda, MD, USA
| | - Reena Karani
- Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Lisa L Willett
- University of Alabama at Birmingham, Birmingham, AL, USA
| |
Collapse
|
24
|
Stolk-Vos AC, Heres MH, Kesteloo J, Verburg D, Hiddema F, Lie DA, de Korne DF. Is there a role for the use of aviation assessment instruments in surgical training preparation? A feasibility study. Postgrad Med J 2016; 93:20-24. [DOI: 10.1136/postgradmedj-2016-133984] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2016] [Revised: 05/12/2016] [Accepted: 05/13/2016] [Indexed: 11/03/2022]
|
25
|
Havyer RD, Nelson DR, Wingo MT, Comfere NI, Halvorsen AJ, McDonald FS, Reed DA. Addressing the Interprofessional Collaboration Competencies of the Association of American Medical Colleges: A Systematic Review of Assessment Instruments in Undergraduate Medical Education. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2016; 91:865-88. [PMID: 26703415 DOI: 10.1097/acm.0000000000001053] [Citation(s) in RCA: 59] [Impact Index Per Article: 7.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/11/2023]
Abstract
PURPOSE To summarize characteristics and validity evidence of tools that assess teamwork in undergraduate medical education (UME), and provide recommendations for addressing the interprofessional collaboration competencies of the Association of American Medical Colleges (AAMC). METHOD The authors conducted a systematic review, searching MEDLINE, MEDLINE In-process, CINAHL, and PsycINFO from January 1, 1979, through April 1, 2014; they searched reference lists and national meeting abstracts. They included original research reports that described a quantitative tool used to assess teamwork in UME. They abstracted characteristics and validity evidence for the tools, plus study quality, according to established frameworks. Two authors independently abstracted 25% of articles and calculated agreement. Authors then applied predefined criteria to identify tools best suited to address the AAMC's teamwork competencies. RESULTS Of 13,549 citations, 70 articles describing 64 teamwork assessment tools were included. Of these 64 tools, 27 (42%) assessed teamwork in classroom, 31 (48%) in simulation, and only 7 (11%) in actual clinical settings. The majority (47; 73%) of tools assessed medical students' teamwork in interprofessional teams. On the basis of content concordance, strength of validity evidence, generalizability of scores, and level of outcomes, four published tools were recommended to assess the AAMC's teamwork competencies: the Collaborative Healthcare Interdisciplinary Relationship Planning Scale, Readiness for Interprofessional Learning Scale, Communication and Teamwork Skills assessment, and Teamwork Mini-Clinical Evaluation Exercise. CONCLUSIONS Substantial validity evidence supports the use of several UME teamwork assessments. Four tools have been appropriately designed and sufficiently studied to constitute appropriate assessments of the AAMC's teamwork competencies.
Collapse
Affiliation(s)
- Rachel D Havyer
- R.D. Havyer is assistant professor of medicine, Mayo Clinic College of Medicine, Rochester, Minnesota. D.R. Nelson is assistant professor of medicine, Mayo Clinic College of Medicine, Rochester, Minnesota. M.T. Wingo is assistant professor of medicine, Mayo Clinic College of Medicine, Rochester, Minnesota. N.I. Comfere is associate professor of dermatology and of laboratory medicine and pathology, Mayo Clinic College of Medicine, Rochester, Minnesota. A.J. Halvorsen is assistant professor of medicine, Mayo Clinic College of Medicine, Rochester, Minnesota. F.S. McDonald is professor of medicine, Mayo Clinic College of Medicine, Rochester, Minnesota, and vice president, Graduate Medical Education, American Board of Internal Medicine, Philadelphia, Pennsylvania. D.A. Reed is associate professor of medicine and of medical education, Mayo Clinic College of Medicine, Rochester, Minnesota
| | | | | | | | | | | | | |
Collapse
|
26
|
Massie J, Ali JM. Workplace-based assessment: a review of user perceptions and strategies to address the identified shortcomings. ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2016; 21:455-73. [PMID: 26003590 DOI: 10.1007/s10459-015-9614-0] [Citation(s) in RCA: 77] [Impact Index Per Article: 9.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/24/2014] [Accepted: 05/13/2015] [Indexed: 05/12/2023]
Abstract
Workplace based assessments (WBAs) are now commonplace in postgraduate medical training. User acceptability and engagement is essential to the success of any medical education innovation. To this end, possessing an insight into trainee and trainer perceptions towards WBAs will help identify the major problems, permitting strategies to be introduced to improve WBA implementation. A review of literature was performed to identify studies examining trainee and trainer perceptions towards WBAs. Studies were excluded if non-English or sampling a non-medical/dental population. The identified literature was synthesised for the purpose of this critical narrative review. It is clear that there is widespread negativity towards WBAs in the workplace. This has negatively impacted on the effectiveness of WBA tools as learning aids. This negativity exists in trainees but also to an extent in their trainers. Insight gained from the literature reveals three dominant problems with WBA implementation: poor understanding as to the purpose of WBAs; insufficient time available for undertaking these assessments; and inadequate training of trainers. Approaches to addressing these three problems with WBA implementation are discussed. It is likely that a variety of solutions will be required. The prevalence of negativity towards WBAs is substantial in both trainees and trainers, eroding the effectiveness of learning that is consequent upon them. The educational community must now listen to the concerns being raised by the users and consider the range of strategies being proposed to improve the experiences of trainees, and their trainers.
Collapse
Affiliation(s)
- Jonathan Massie
- School of Clinical Medicine, University of Cambridge, Cambridge, UK
| | - Jason M Ali
- Department of Surgery, University of Cambridge, BOX 202, Addenbrookes Hospital, Cambridge, CB2 0QQ, UK.
| |
Collapse
|
27
|
Carraccio C, Englander R, Van Melle E, Ten Cate O, Lockyer J, Chan MK, Frank JR, Snell LS. Advancing Competency-Based Medical Education: A Charter for Clinician-Educators. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2016; 91:645-9. [PMID: 26675189 DOI: 10.1097/acm.0000000000001048] [Citation(s) in RCA: 192] [Impact Index Per Article: 24.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/18/2023]
Abstract
The International Competency-Based Medical Education (ICBME) Collaborators have been working since 2009 to promote understanding of competency-based medical education (CBME) and accelerate its uptake worldwide. This article presents a charter, supported by a literature-based rationale, which is meant to provide a shared mental model of CBME that will serve as a path forward in its widespread implementation.At a 2013 summit, the ICBME Collaborators laid the groundwork for this charter. Here, the fundamental principles of CBME and professional responsibilities of medical educators in its implementation process are described. The authors outline three fundamental principles: (1) Medical education must be based on the health needs of the populations served; (2) the primary focus of education and training should be the desired outcomes for learners rather than the structure and process of the educational system; and (3) the formation of a physician should be seamless across the continuum of education, training, and practice.Building on these principles, medical educators must demonstrate commitment to teaching, assessing, and role modeling the range of identified competencies. In the clinical setting, they must provide supervision that balances patient safety with the professional development of learners, being transparent with stakeholders about level of supervision needed. They must use effective and efficient assessment strategies and tools for basing transition decisions on competence rather than time in training, empowering learners to be active participants in their learning and assessment. Finally, advancing CBME requires program evaluation and research, faculty development, and a collaborative approach to realize its full potential.
Collapse
Affiliation(s)
- Carol Carraccio
- C. Carraccio is vice president, Competency-Based Assessment, American Board of Pediatrics, Chapel Hill, North Carolina. R. Englander was senior director of competency-based learning and assessment, Association of American Medical Colleges, Washington, DC, at the time this was written. E. Van Melle is education researcher, Queen's University, Kingston, Ontario, Canada, and education scientist, Royal College of Physicians and Surgeons of Canada, Ottawa, Ontario, Canada. O. ten Cate is professor of medical education and director, Center for Research and Development of Education, University Medical Center, Utrecht, the Netherlands. J. Lockyer is senior associate dean-education and professor, Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada. M.-K. Chan is associate professor, Department of Pediatrics and Child Health, University of Manitoba, Winnipeg, Manitoba, Canada, and clinician educator, CanMEDS & Faculty Development, Royal College of Physicians and Surgeons of Canada, Ottawa, Ontario, Canada. J.R. Frank is director, Specialty Education, Strategy, and Standards, Office of Specialty Education, Royal College of Physicians and Surgeons of Canada, and director of educational research and development, Department of Emergency Medicine, University of Ottawa, Ottawa, Ontario, Canada. L.S. Snell is professor of medicine, Centre for Medical Education, McGill University, Montreal, Quebec, Canada, and senior clinician educator, Royal College of Physicians and Surgeons of Canada, Ottawa, Ontario, Canada
| | | | | | | | | | | | | | | |
Collapse
|
28
|
Wingo MT, Havyer RDA, Comfere NI, Nelson DR, Reed DA. Interprofessional collaboration milestones: advocating for common assessment criteria in graduate medical education. BMC MEDICAL EDUCATION 2015; 15:149. [PMID: 26369325 PMCID: PMC4570478 DOI: 10.1186/s12909-015-0432-0] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/11/2015] [Accepted: 09/04/2015] [Indexed: 05/14/2023]
Abstract
BACKGROUND Milestone-based assessments of resident physicians inform critical decisions regarding resident competence and advancement. Thus, it is essential that milestone evaluations are based upon strong validity evidence and that consistent evaluation criteria are used across residency programs. A common approach to assessment of interprofessional collaboration milestones is particularly important since standardized measures of individual resident competence in interprofessional collaboration have not been established. DISCUSSION We propose that assessments of interprofessional collaboration in graduate medical education meet common criteria, namely, these assessments should: 1) measure competency of an individual resident, 2) occur in the context of an interprofessional team, 3) be ascertained via direct observation of the resident, 4) be performed in a real-world clinical practice setting (such as a hospital ward, outpatient clinic, or operating room). We present the evidence-based rationale for these criteria and cite examples of published assessment instruments that fulfill one or more of the criteria, however further research is needed to ensure fidelity of assessments. The proposed criteria may assist residency educators as they endeavor to provide robust and consistent assessments of interprofessional collaboration milestones.
Collapse
Affiliation(s)
- Majken T Wingo
- Division of Primary Care Internal Medicine, Department of Medicine, Mayo Clinic, Rochester, MN, USA.
| | - Rachel D A Havyer
- Division of Primary Care Internal Medicine, Department of Medicine, Mayo Clinic, Rochester, MN, USA.
| | - Nneka I Comfere
- Departments of Dermatology and Laboratory Medicine & Pathology, Mayo Clinic, Rochester, MN, USA.
| | - Darlene R Nelson
- Division of Pulmonary and Critical Care Medicine, Department of Medicine, Mayo Clinic, Rochester, MN, USA.
| | - Darcy A Reed
- Division of Primary Care Internal Medicine, Department of Medicine, Mayo Clinic, Rochester, MN, USA.
| |
Collapse
|
29
|
Olupeliyawa A, Balasooriya C. The impact of programmatic assessment on student learning: what can the students tell us? MEDICAL EDUCATION 2015; 49:453-456. [PMID: 25924119 DOI: 10.1111/medu.12730] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/04/2023]
|
30
|
Thistlethwaite J, Dallest K. Interprofessional teamwork: still haven't decided what we are educating for? MEDICAL EDUCATION 2014; 48:556-558. [PMID: 24807430 DOI: 10.1111/medu.12471] [Citation(s) in RCA: 18] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/03/2023]
|