1
|
Ark T, Kalet A, Tewksbury L, Altshuler L, Crowe R, Wilhite J, Hardowar K, Zabar S, Gillespie C. Validity evidence for the clinical communication skills assessment tool (CCSAT) from 9 years of implementation in a high stakes medical student OSCE. PATIENT EDUCATION AND COUNSELING 2024; 127:108323. [PMID: 38851013 DOI: 10.1016/j.pec.2024.108323] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/08/2023] [Revised: 03/22/2024] [Accepted: 05/13/2024] [Indexed: 06/10/2024]
Abstract
OBJECTIVES Communication and other clinical skills are routinely assessed in medical schools using Objective Structured Clinical Examinations (OSCEs) so routinely that it can be difficult to monitor and maintain validity. We report on the accumulation of validity evidence for the Clinical Communication Skills Assessment Tool (CCSAT) based on its use with 9 cohorts of medical students in a high stakes OSCE. METHODS We describe the implementation of the CCSAT including information on the underlying model, the tool's items, domains, scales and scoring, and its role in curriculum. Internal structure is explored through item, internal consistency, and confirmatory factor analyses. Evidence for CCSAT validity is synthesized within prevailing frameworks (Messick12 and Kane13) based on continuous quality improvement and use of the CCSAT for feedback, remediation, curricular design, and research. RESULTS Implementation of the CCSAT over time has facilitated our communication skills curriculum and training. Thoughtful case development and investment in standardized patient training has contributed to data quality. Item analysis supports our behaviorally anchored scale (not done, partly and well done) and the skills domains suggested by an a priori evidence-based clinical communication model were confirmed via analysis of actual student data. Evidence synthesized across the frameworks suggests consistent validity of the CCSAT for generalization inferences (that it captures the construct), responsiveness (sensitivity to change/difference), content validity/internal structure, relationships to other variables, and consequences/implications. More evidence is needed to strengthen validity of CCSAT scores for understanding extrapolation inferences and real-world implications. CONCLUSIONS AND PRACTICE IMPLICATIONS This pragmatic approach to evaluating validity within a program of assessment serves as a model for medical schools seeking to continuously monitor the quality of clinical skill assessments, a need made particularly relevant since the US NBME no longer requires the Step 2 Clinical Skills exam, leaving individual schools with the responsibility for ensuring graduates have acquired the requisite core clinical skills. We document strong evidence for CCSAT validity over time and across cohorts as well as areas for improvement and further examination.
Collapse
Affiliation(s)
- Tavinder Ark
- Medical College of Wisconsin, 8701 Watertown Plank Road, Milwaukee, WI 53226, USA
| | - Adina Kalet
- Medical College of Wisconsin, 8701 Watertown Plank Road, Milwaukee, WI 53226, USA
| | - Linda Tewksbury
- NYU Grossman School of Medicine, NYU Langone Health, 462 1st Avenue, New York, NY 10016, USA
| | - Lisa Altshuler
- NYU Grossman School of Medicine, NYU Langone Health, 462 1st Avenue, New York, NY 10016, USA
| | - Ruth Crowe
- NYU Long Island School of Medicine, NYU Langone Health, 101 Mineola Blvd, Mineola, NY 11501, USA
| | - Jeffrey Wilhite
- NYU Grossman School of Medicine, NYU Langone Health, 462 1st Avenue, New York, NY 10016, USA.
| | - Khemraj Hardowar
- NYU Grossman School of Medicine, NYU Langone Health, 462 1st Avenue, New York, NY 10016, USA
| | - Sondra Zabar
- NYU Grossman School of Medicine, NYU Langone Health, 462 1st Avenue, New York, NY 10016, USA
| | - Colleen Gillespie
- NYU Grossman School of Medicine, NYU Langone Health, 462 1st Avenue, New York, NY 10016, USA; Institute for Innovations in Medical Education, NYU Grossman School of Medicine, 462 1st Avenue, New York, NY 10016, USA
| |
Collapse
|
2
|
Altshuler L, Wilhite JA, Hardowar K, Crowe R, Hanley K, Kalet A, Zabar S, Gillespie C, Ark T. Understanding medical student paths to communication skills expertise using latent profile analysis. MEDICAL TEACHER 2023; 45:1140-1147. [PMID: 36961759 DOI: 10.1080/0142159x.2023.2193303] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/18/2023]
Abstract
PURPOSE To describe patterns of clinical communication skills that inform curriculum enhancement and guide coaching of medical students. MATERIALS AND METHODS Performance data from 1182 consenting third year medical students in 9 cohorts (2011-2019), on a 17-item Clinical Communication Skills Assessment Tool (CCSAT) completed by trained Standardized Patients as part of an eight case high stakes Comprehensive Clinical Skills Exam (CCSE) were analyzed using latent profile analysis (LPA). Assessment domains included: information gathering (6 items), relationship development (5 items), patient education (3 items), and organization/time management (3 items). LPA clustered learners with similar strength/weakness into profiles based on item response patterns across cases. One-way analysis of variance (ANOVA) assessed for significant differences by profile for CCSAT items. RESULTS Student performance clustered into six profiles in three groups, high performing (HP1 and HP2-Low Patient Education, 15.7%), average performing (AP1 and AP2-Interrupters, 40.9%), and lower performing profiles (LP1-Non-interrupters and LP2, 43.4%) with adequate model fit estimations and similar distribution in each cohort. We identified 3 CCSAT items that discriminated among learner's skill profiles. CONCLUSION Clinical communication skill performance profiles provide nuanced, benchmarked guidance for curriculum improvement and tailoring of communication skills coaching.
Collapse
Affiliation(s)
- Lisa Altshuler
- NYU Grossman School of Medicine, NYU Langone Health, New York, NY, USA
| | - Jeffrey A Wilhite
- NYU Grossman School of Medicine, NYU Langone Health, New York, NY, USA
| | - Khemraj Hardowar
- NYU Grossman School of Medicine, NYU Langone Health, New York, NY, USA
| | - Ruth Crowe
- NYU Grossman School of Medicine, NYU Langone Health, New York, NY, USA
| | - Kathleen Hanley
- NYU Grossman School of Medicine, NYU Langone Health, New York, NY, USA
| | - Adina Kalet
- Kern Institute, Medical College of Wisconsin, Milwaukee, WI, USA
| | - Sondra Zabar
- NYU Grossman School of Medicine, NYU Langone Health, New York, NY, USA
| | - Colleen Gillespie
- Institute for Innovations in Medical Education, NYU Grossman School of Medicine, New York, NY, USA
| | - Tavinder Ark
- Kern Institute, Medical College of Wisconsin, Milwaukee, WI, USA
| |
Collapse
|
3
|
Coble C, Srivastav S, Glick A, Bradshaw C, Osman C. Teaching SSHADESS Versus HEADSS to Medical Students: An Association With Improved Communication Skills and Increased Psychosocial Factor Assessments. Acad Pediatr 2023; 23:209-215. [PMID: 36130691 DOI: 10.1016/j.acap.2022.09.012] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/01/2022] [Revised: 08/10/2022] [Accepted: 09/13/2022] [Indexed: 01/19/2023]
Abstract
OBJECTIVE The most recent Bright Futures edition describes both the HEADSS (Home, Education, Drugs, Sexuality, Safety) and the strength-based SSHADESS (Strengths, School, Home, Activities, Drugs, Emotions, Sexuality, Safety) frameworks for conducting an adolescent psychosocial history. We found limited research comparing the effectiveness of these 2 frameworks. Our study objective was to examine whether teaching medical students SSHADESS versus HEADSS is associated with increased communication skills and/or completion of the psychosocial assessment. METHODS In this retrospective cohort study of pediatric clerkship students at New York University Grossman School of Medicine, we collected data from observer (faculty, fellow, and resident) and standardized patient (SP) assessments during Objective Structured Clinical Examinations. Primary outcomes were observer and SP-rated usage of communication skills. Secondary outcomes were observer-rated assessment of 6 psychosocial factors. Our predictor variable was whether students were taught HEADSS (11/2015-10/2016) or SSHADESS (11/2016-10/2017). We used Fisher's exact tests and then logistic regressions to adjust for pediatrics clerkship timing and baseline communication skills. RESULTS About 200 students were assessed (n = 97 HEADSS cohort, n = 103 SSHADESS cohort). In adjusted analyses of observer scores, the SSHADESS cohort was more likely to use all communication skills (adjusted odds ratio [aOR] 3.2, 95% confidence interval [CI] 1.7-6.3]) and assess all psychosocial factors (aOR 1.9, 95% CI 1.01-3.4). There was no significant difference in SP communication scores. DISCUSSION Teaching SSHADESS was associated with higher observer-rated communication skills scores and improved completeness in assessment of psychosocial factors. Future work should examine the efficacy of SSHADESS through workplace-based assessments and 360 degree assessments from adolescent patients.
Collapse
Affiliation(s)
- Chanelle Coble
- Department of Pediatrics, NYU Grossman School of Medicine (C Coble, A Glick, C Bradshaw, and C Osman), New York, NY.
| | - Shivani Srivastav
- Department of Pediatrics, UCSF Benioff Children's Hospital Oakland (S Srivastav), Oakland, Calif
| | - Alexander Glick
- Department of Pediatrics, NYU Grossman School of Medicine (C Coble, A Glick, C Bradshaw, and C Osman), New York, NY
| | - Chanda Bradshaw
- Department of Pediatrics, NYU Grossman School of Medicine (C Coble, A Glick, C Bradshaw, and C Osman), New York, NY
| | - Cynthia Osman
- Department of Pediatrics, NYU Grossman School of Medicine (C Coble, A Glick, C Bradshaw, and C Osman), New York, NY
| |
Collapse
|
4
|
Engel KG, Pedersen K, Johansen MD, Schoennemann KR, Kjaer LB, Nayahangan LJ. Consensus on communication curriculum content in Danish undergraduate medical education: A Delphi study. MEDICAL TEACHER 2022; 44:1221-1227. [PMID: 35649701 DOI: 10.1080/0142159x.2022.2072280] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
BACKGROUND The acquisition of skills in patient-centered communication is a critical aspect of medical education which demands both resource-intensive instruction and longitudinal opportunities for learning. Significant variation currently exists in the content and timing of communication education. The aim of this study was to establish consensus regarding communication curriculum content for undergraduate medical education (UME) within the country of Denmark. METHODS This study employed a Delphi process which is a widely accepted method for establishing consensus among experts and can be utilized to guide planning and decision-making in education. For this study, consensus was based on greater than 60% agreement between participants. Diverse stakeholders, representing all four universities with medical schools in Denmark, participated in an iterative three-round Delphi process which involved: (1) identifying key curricular elements for medical student education, (2) rating the importance of each item, and (3) prioritizing items relative to one another and rating each item based on the level of mastery that was expected for each skill (i.e. knowledge, performance with supervision, or performance independently). RESULTS A national sample of 149 stakeholders participated with a 70% response rate for round 1, 81% for round 2, and 86% for round 3. The completed Delphi process yielded 56 content items which were prioritized in rank order lists within five categories: (1) establishing rapport, engaging patient perspectives and responding to needs; (2) basic communication skills and techniques; (3) phases and structure of the encounter; (4) personal characteristics and skills of the student; (5) specific challenging patient groups and context-dependent situations. DISCUSSION Using a Delphi process, it was possible to achieve consensus regarding communication curriculum content for UME. These findings provide an important foundation for ensuring greater uniformity in UME, as well as supporting the important longitudinal goals of communication skill development across medical training.
Collapse
Affiliation(s)
- Kirsten Greineder Engel
- Copenhagen Academy for Medical Education and Simulation (CAMES), Centre for HR and Education and the University of Copenhagen, Copenhagen, Denmark
- Massachusetts General Hospital, Boston, MA, USA
| | - Kamilla Pedersen
- Centre for Educational Development, Aarhus University, Aarhus, Denmark
| | - Mette Dencker Johansen
- Department of Health Science and Technology, Faculty of Medicine, Aalborg University, Aalborg, Denmark
| | - Katrine Rahbek Schoennemann
- Department of Oncology, Odense University Hospital, Odense, Denmark
- University of Southern Denmark, Odense, Denmark
| | - Louise Binow Kjaer
- Centre for Educational Development, Aarhus University, Aarhus, Denmark
- Health, and Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
| | - Leizl Joy Nayahangan
- Copenhagen Academy for Medical Education and Simulation (CAMES), Centre for HR and Education and the University of Copenhagen, Copenhagen, Denmark
| |
Collapse
|
5
|
Cline C, Santuzzi AM, Samonds KE, LaDue N, Bergan-Roller HE. Assessing how students value learning communication skills in an undergraduate anatomy and physiology course. ANATOMICAL SCIENCES EDUCATION 2022; 15:1032-1044. [PMID: 34665527 DOI: 10.1002/ase.2144] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/01/2021] [Revised: 10/15/2021] [Accepted: 10/16/2021] [Indexed: 06/13/2023]
Abstract
Students, particularly those in science, technology, engineering and mathematics (STEM) and healthcare-related programs, should develop proficient interpersonal skills, including communication. To help students develop effective communication skills, instructors need to consider the value students give to learning these skills. The Student Attitudes Toward Communication Skills Survey (SATCSS) was developed to measure how undergraduate students value learning communication skills based on Expectancy-Value Theory across three modes of communication (verbal, written, non-verbal). The survey was given to students interested in healthcare professions and enrolled in an undergraduate anatomy and physiology (A&P) course (n = 233) at a Midwest research active university. The survey showed evidence of validity, measuring two components: (1) "Value to Profession" (attainment and utility value) and (2) "Value to Self" (intrinsic value and cost). There was a significant difference in sub-scores among the four task values such that students thought that learning communication skills was important and relevant (high attainment and utility value) but not interesting (low intrinsic value) and costly. Students with high total scores valued communication skills across all four task values. As total value scores decreased, it was first due to students finding learning communication skills to be time prohibitive and then a lack of interest in learning communication skills. Based on these results, it is recommended that instructors incorporate communication skills training into content that is already part of their A&P course to reduce time concerns. Additional recommendations include using reflective activities and humor to increase student interest.
Collapse
Affiliation(s)
- Christina Cline
- Department of Biological Sciences, College of Liberal Arts and Sciences, Northern Illinois University, DeKalb, Illinois, USA
| | - Alecia M Santuzzi
- Department of Psychology, College of Liberal Arts and Sciences, Northern Illinois University, DeKalb, Illinois, USA
| | - Karen E Samonds
- Department of Biological Sciences, College of Liberal Arts and Sciences, Northern Illinois University, DeKalb, Illinois, USA
| | - Nicole LaDue
- Department of Geology and Environmental Geosciences, College of Liberal Arts and Sciences, Northern Illinois University, DeKalb, Illinois, USA
| | - Heather E Bergan-Roller
- Department of Biological Sciences, College of Liberal Arts and Sciences, Northern Illinois University, DeKalb, Illinois, USA
| |
Collapse
|
6
|
Altshuler L, Ark T, Wilhite J, Hardowar K, Crowe R, Hanley K, L Kalet A, Zabar S, Gillespie C. Using Latent Profile Analysis to Describe and Understand Medical Student Paths to Communication Skills Expertise. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2022; 97:S151. [PMID: 37838883 DOI: 10.1097/acm.0000000000004814] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/20/2023]
Affiliation(s)
- Lisa Altshuler
- Author affiliations: L. Altshuler, J. Wilhite, K. Hardowar, R. Crowe, K. Hanley, S. Zabar, C. Gillespie, NYU Grossman School of Medicine; T. Ark, A.L. Kalet, Medical College of Wisconsin
| | | | | | | | | | | | | | | | | |
Collapse
|
7
|
Fisher H, Zabar S, Chodosh J, Langford A, Trinh-Shevrin C, Sherman S, Altshuler L. A novel simulation-based approach to training for recruitment of older adults to clinical trials. BMC Med Res Methodol 2022; 22:180. [PMID: 35764920 PMCID: PMC9238219 DOI: 10.1186/s12874-022-01643-4] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/13/2021] [Accepted: 05/18/2022] [Indexed: 01/30/2023] Open
Abstract
BACKGROUND The need to engage adults, age 65 and older, in clinical trials of conditions typical in older populations, (e.g. hypertension, diabetes mellitus, Alzheimer's disease and related dementia) is exponentially increasing. Older adults have been markedly underrepresented in clinical trials, often exacerbated by exclusionary study criteria as well as functional dependencies that preclude participation. Such dependencies may further exacerbate communication challenges. Consequently, the evidence of what works in subject recruitment is less generalizable to older populations, even more so for those from racial and ethnic minority and low-income communities. METHODS To support capacity of research staff, we developed a virtual, three station simulation (Group Objective Structured Clinical Experience-GOSCE) to teach research staff communication skills. This 2-h course included a discussion of challenges in recruiting older adults; skills practice with Standardized Participants (SPs) and faculty observer who provided immediate feedback; and debrief to highlight best practices. Each learner had opportunities for active learning and observational learning. Learners completed a retrospective pre-post survey about the experience. SP completed an 11-item communication checklist evaluating the learner on a series of established behaviorally anchored communication skills (29). RESULTS In the research staff survey, 92% reported the overall activity taught them something new; 98% reported it provided valuable feedback; 100% said they would like to participate again. In the SP evaluation there was significant variation: the percent well-done of items by case ranged from 25-85%. CONCLUSIONS Results from this pilot suggest that GOSCEs are a (1) acceptable; (2) low cost; and (3) differentiating mechanism for training and assessing research staff in communication skills and structural competency necessary for participant research recruitment.
Collapse
Affiliation(s)
- Harriet Fisher
- Division of General Internal Medicine and Clinical Innovation, NYU Grossman School of Medicine (NYUGSOM), New York, USA.
| | - Sondra Zabar
- grid.137628.90000 0004 1936 8753Division of General Internal Medicine and Clinical Innovation, NYU Grossman School of Medicine (NYUGSOM), New York, USA
| | - Joshua Chodosh
- grid.137628.90000 0004 1936 8753Department of Population Health, NYU Grossman School of Medicine (NYUGSOM), New York, USA ,grid.137628.90000 0004 1936 8753Division of Geriatric Medicine, NYU Grossman School of Medicine (NYUGSOM), New York, USA
| | - Aisha Langford
- grid.137628.90000 0004 1936 8753Department of Population Health, NYU Grossman School of Medicine (NYUGSOM), New York, USA
| | - Chau Trinh-Shevrin
- grid.137628.90000 0004 1936 8753Department of Population Health, NYU Grossman School of Medicine (NYUGSOM), New York, USA
| | - Scott Sherman
- grid.137628.90000 0004 1936 8753Department of Population Health, NYU Grossman School of Medicine (NYUGSOM), New York, USA
| | - Lisa Altshuler
- grid.137628.90000 0004 1936 8753Division of General Internal Medicine and Clinical Innovation, NYU Grossman School of Medicine (NYUGSOM), New York, USA
| |
Collapse
|
8
|
Gärtner J, Bußenius L, Schick K, Prediger S, Kadmon M, Berberat PO, Harendza S. Validation of the ComCare index for rater-based assessment of medical communication and interpersonal skills. PATIENT EDUCATION AND COUNSELING 2022; 105:1004-1008. [PMID: 34389227 DOI: 10.1016/j.pec.2021.07.051] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/29/2021] [Revised: 07/12/2021] [Accepted: 07/29/2021] [Indexed: 06/13/2023]
Abstract
OBJECTIVES To develop and validate a short instrument to assess undergraduate medical students' communication and interpersonal skills in videographed history taking situations with simulated patients. METHODS Sixty-seven undergraduate medical students participating in an assessment including videographed physician-patient encounters for history taking with five simulated patients were included in this study. The last video of each participant's consultation hour was rated by two independent assessors with the eight-item ComCare index for assessment of communication and interpersonal skills newly designed for the external rater perspective (ComCareR). We compared the sum scores of the ComCareR with ratings of the same videos with the Kalamazoo Communication Skills Assessment Form from an observational perspective (KCSAFd-video) and the Global Rating scale (GR), which also measure communication and interpersonal skills. RESULTS The ComCareR showed an excellent interrater reliability (ICC = .85). We found a small but significant correlation with the KCSAFd-video Interpersonal Competence (ρ = .34, 95% CI [.10,.54]) and a high positive correlation with the GR (ρ = .59, 95% CI [.40,.73]). CONCLUSIONS The ComCareR is a valid and brief index for holistic assessment of communication and interpersonal skills in physician-patient encounters. PRACTICE IMPLICATIONS The ComCareR can be used for quick rater-based assessment of physicians' communication and interpersonal skills.
Collapse
Affiliation(s)
- Julia Gärtner
- III. Department of Internal Medicine, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Lisa Bußenius
- Department of Biochemistry and Molecular Cell Biology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Kristina Schick
- TUM Medical Education Center, Technical University of Munich, Munich, Germany
| | - Sarah Prediger
- III. Department of Internal Medicine, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Martina Kadmon
- Faculty of Medicine, University of Augsburg, Augsburg, Germany
| | - Pascal O Berberat
- TUM Medical Education Center, Technical University of Munich, Munich, Germany
| | - Sigrid Harendza
- III. Department of Internal Medicine, University Medical Center Hamburg-Eppendorf, Hamburg, Germany.
| |
Collapse
|
9
|
Howell HB, Desai PV, Altshuler L, McGrath M, Ramsey R, Vrablik L, Levy FH, Zabar S. Teaching and Assessing Communication Skills in Pediatric Residents: How Do Parents Think We Are Doing? Acad Pediatr 2022; 22:179-183. [PMID: 34186252 DOI: 10.1016/j.acap.2021.06.011] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/22/2021] [Revised: 06/14/2021] [Accepted: 06/21/2021] [Indexed: 11/28/2022]
Abstract
OBJECTIVE Curricula designed to teach and assess the communication skills of pediatric residents variably integrates the parent perspective. We compared pediatric residents' communication skills in an objective structured clinical exam (OSCE) case as assessed by Family Faculty (FF), parents of pediatric patients, versus standardized patients (SP). METHODS Residents participated in an OSCE case with a SP acting as a patient's parent. We compared resident performance as assessed by FF and SP with a behaviorally-anchored checklist. Items were rated as not done, partly done or well done, with well-done indicating mastery. The residents evaluated the experience. RESULTS 42 residents consented to study participation. FF assessed a lower percentage of residents as demonstrating skill mastery as compared to SP in 19 of the 23 behaviors. There was a significant difference between FF and SP for Total Mastery Score and Mastery of the Competency Scores in three domains (Respect and Value, Information Sharing and Participation in Care and Decision Making). The majority of residents evaluated the experience favorably. CONCLUSION Involving parents of pediatric patients in the instructive and assessment components of a communication curriculum for pediatric residents adds a unique perspective and integrates the true stakeholders in parent-physician communication.
Collapse
Affiliation(s)
- Heather B Howell
- Department of Pediatrics, New York University Grossman School of Medicine (HB Howell, PV Desai, L Vrablik, and FH Levy), New York, NY.
| | - Purnahamsi V Desai
- Department of Pediatrics, New York University Grossman School of Medicine (HB Howell, PV Desai, L Vrablik, and FH Levy), New York, NY
| | - Lisa Altshuler
- Department of Medicine, New York University Grossman School of Medicine (L Altshuler and S Zabar), New York, NY
| | - Meaghan McGrath
- Sala Institute for Child and Family Centered Care, Hassenfeld Children's Hospital (M McGrath, R Ramsey and FH Levy), New York, NY
| | - Rachel Ramsey
- Sala Institute for Child and Family Centered Care, Hassenfeld Children's Hospital (M McGrath, R Ramsey and FH Levy), New York, NY
| | - Lauren Vrablik
- Department of Pediatrics, New York University Grossman School of Medicine (HB Howell, PV Desai, L Vrablik, and FH Levy), New York, NY
| | - Fiona H Levy
- Department of Pediatrics, New York University Grossman School of Medicine (HB Howell, PV Desai, L Vrablik, and FH Levy), New York, NY; Sala Institute for Child and Family Centered Care, Hassenfeld Children's Hospital (M McGrath, R Ramsey and FH Levy), New York, NY
| | - Sondra Zabar
- Department of Medicine, New York University Grossman School of Medicine (L Altshuler and S Zabar), New York, NY
| |
Collapse
|
10
|
Kalet A, Ark TK, Monson V, Song HS, Buckvar-Keltz L, Harnik V, Yingling S, Rivera R, Tewksbury L, Lusk P, Crowe R. Does a measure of Medical Professional Identity Formation predict communication skills performance? PATIENT EDUCATION AND COUNSELING 2021; 104:3045-3052. [PMID: 33896685 DOI: 10.1016/j.pec.2021.03.040] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/20/2020] [Revised: 03/05/2021] [Accepted: 03/31/2021] [Indexed: 06/12/2023]
Abstract
OBJECTIVE To validate an approach to measuring professional identity formation (PIF), we explore if the Professional Identity Essay (PIE), a stage score measure of medical professional identity (PI), predicts clinical communication skills. METHODS Students completed the PIE during medical school orientation and a 3-case Objective Structured Clinical Exam (OSCE) where standardized patients reliably assessed communication skills in 5 domains. Using mediation analyses, relationships between PIE stage scores and communication skills were explored. RESULTS For the 351 (89%) consenting students, controlling for individual characteristics, there were increases in patient counseling (6.5%, p<0.01), information gathering (4.3%, p = 0.01), organization and management (4.1%, p = 0.02), patient assessment (3.6%, p = 0.04), and relationship development (3.5%, p = 0.03) skills for every half stage increase in PIE score. The communication skills of lower socio-economic status (SES) students are indirectly impacted by their slightly higher PIE stage scores. CONCLUSION Higher PIE stage scores are associated with higher communication skills and lower SES. PRACTICE IMPLICATIONS PIE predicts critical clinical skills and identifies how SES and other characteristics indirectly impact future clinical performance, providing validity evidence for using PIE as a tool in longitudinal formative academic coaching, program and curriculum evaluation, and research.
Collapse
Affiliation(s)
- Adina Kalet
- Department of Medicine, NYU School of Medicine, New York, USA; Program on Medical Education and Technology, NYU School of Medicine, New York, USA; Robert D. and Patricia E. Kern Institute for the Transformation of Medical Education, Medical College of Wisconsin.
| | - Tavinder K Ark
- School of Public Health, University of British Columbia, Vancouver, Canada; Robert D. and Patricia E. Kern Institute for the Transformation of Medical Education, Medical College of Wisconsin
| | - Verna Monson
- Private Consultant; Robert D. and Patricia E. Kern Institute for the Transformation of Medical Education, Medical College of Wisconsin
| | - Hyuksoon S Song
- Department of Education, Georgian Court University, Lakewood, USA
| | - Lynn Buckvar-Keltz
- Department of Medicine, NYU School of Medicine, New York, USA; Office of Student Affairs, NYU School of Medicine, New York, USA; Program on Medical Education and Technology, NYU School of Medicine, New York, USA
| | - Victoria Harnik
- Office of Medical Education, NYU School of Medicine, New York, USA
| | - Sandra Yingling
- Department of Medical Education, University of Illinois-Chicago, Chicago, USA
| | - Rafael Rivera
- Office of Admissions and Financial Aid, NYU School of Medicine, New York, USA
| | - Linda Tewksbury
- Office of Student Affairs, NYU School of Medicine, New York, USA
| | - Penelope Lusk
- Department of Medicine, NYU School of Medicine, New York, USA
| | - Ruth Crowe
- Office of Medical Education, NYU School of Medicine, New York, USA
| |
Collapse
|
11
|
Forbes R, Clasper B, Ilango A, Kan H, Peng J, Mandrusiak A. Effectiveness of patient education training on health professional student performance: A systematic review. PATIENT EDUCATION AND COUNSELING 2021; 104:2453-2466. [PMID: 33678497 DOI: 10.1016/j.pec.2021.02.039] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/27/2020] [Revised: 01/01/2021] [Accepted: 02/21/2021] [Indexed: 06/12/2023]
Abstract
BACKGROUND Despite the fundamental role of patient education, concerns have been raised by health professionals, students, and professional bodies regarding student preparation for this area of practice. OBJECTIVE To evaluate the effectiveness of patient education training on health professional student performance. METHOD A systematic search was performed across PubMed, Cinahl, Embase, and Cochrane Library. Reference and forward citation searches of included articles were conducted. Inclusion criteria were published journal articles from 2010 to 2020 regarding patient education training for health professional students with measures relating to patient education performance. Quality appraisal of individual studies was conducted using Cochrane risk-of-bias v2 or ROBINS-I; overall appraisal was also determined. Sixteen articles met eligibility criteria. Independent data extraction was undertaken. RESULTS Fourteen studies reported improvements in student performance following patient education training; two studies reported no significant improvements. Of different modalities examined, simulation-based learning appeared to be most effective. CONCLUSION Patient education training enhances health professional students' performance of patient education. Future research is needed to evaluate effectiveness of pedagogically informed interventions across health professions using objective approaches to assess student performance in clinically relevant settings. PRACTICAL VALUE This review supports the use of specific training approaches to improve student performance of patient education.
Collapse
Affiliation(s)
- Roma Forbes
- School of Health and Rehabilitation Sciences, The University of Queensland, Brisbane, QLD, 4072, Australia.
| | - Bennita Clasper
- School of Health and Rehabilitation Sciences, The University of Queensland, Brisbane, QLD, 4072, Australia
| | - Anusha Ilango
- School of Health and Rehabilitation Sciences, The University of Queensland, Brisbane, QLD, 4072, Australia
| | - Haeley Kan
- School of Health and Rehabilitation Sciences, The University of Queensland, Brisbane, QLD, 4072, Australia
| | - Julia Peng
- School of Health and Rehabilitation Sciences, The University of Queensland, Brisbane, QLD, 4072, Australia
| | - Allison Mandrusiak
- School of Health and Rehabilitation Sciences, The University of Queensland, Brisbane, QLD, 4072, Australia
| |
Collapse
|
12
|
Objective Structured Clinical Examinations as a Method of Competency Evaluation in a Primary Care Nurse Practitioner Program. Nurse Educ 2020; 46:317-321. [PMID: 33315703 DOI: 10.1097/nne.0000000000000951] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
BACKGROUND Objective structured clinical examinations (OSCEs) are an effective method of instruction and evaluation. Their use in nurse practitioner (NP) programs as a means of evaluation warrants further exploration. PURPOSE The purpose of this project was to test the applicability and feasibility of implementation of OSCEs as a means of formative evaluation in a primary care NP program and to determine student feedback about the experience. METHODS This project evaluated the implementation of an educational intervention using 2 previously developed and tested OSCE scenarios with standardized patients. RESULTS Mean scores were computed for the subcategories, as well as for global rating scores on the OSCE. Strong correlations were found across categories within the scenario; no correlations were found between scenarios. Moderate intraclass correlation coefficients were achieved. Student feedback about the experience, including perceived competence and confidence, was overwhelmingly positive. CONCLUSIONS This project demonstrates success of a pilot OSCE program.
Collapse
|
13
|
Sartori DJ, Hayes RW, Horlick M, Adams JG, Zabar SR. The TeleHealth OSCE: Preparing Trainees to Use Telemedicine as a Tool for Transitions of Care. J Grad Med Educ 2020; 12:764-768. [PMID: 33391602 PMCID: PMC7771608 DOI: 10.4300/jgme-d-20-00039.1] [Citation(s) in RCA: 21] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/13/2020] [Revised: 04/26/2020] [Accepted: 08/02/2020] [Indexed: 12/25/2022] Open
Abstract
BACKGROUND Telemedicine holds promise to bridge the transition of care between inpatient and outpatient settings. Despite this, the unique communication and technical skills required for virtual encounters are not routinely taught or practiced in graduate medical education (GME) programs. OBJECTIVE To develop an objective structured clinical examination (OSCE) case to assess residents' telemedicine-specific skills and identify potential gaps in our residency program's curriculum. METHODS As part of a multi-station OSCE in 2019, we developed a case simulating a remote encounter between a resident and a recently discharged standardized patient. We developed an assessment tool comprising specific behaviors anchored to "not done," "partly done," and "well done" descriptors to evaluate core communication and telemedicine-specific skills. RESULTS Seventy-eight NYU internal medicine residents participated in the case. Evaluations from 100% of participants were obtained. Residents performed well in Information Gathering and Relationship Development domains. A mean 95% (SD 3.3%) and 91% (SD 4.9%) of residents received "well done" evaluations across these domains. A mean 78% (SD 14%) received "well done" within Education/Counseling domain. However, only 46% (SD 45%) received "well done" evaluations within the Telemedicine domain; specific weak areas included performing a virtual physical examination (18% well done) and leveraging video to augment history gathering (17% well done). There were no differences in telemedicine-specific skill evaluations when stratified by training track or postgraduate year. CONCLUSIONS We simulate a post-discharge virtual encounter and present a novel assessment tool that uncovers telemedicine-specific knowledge gaps in GME trainees.
Collapse
Affiliation(s)
- Daniel J Sartori
- Assistant Professor, Department of Medicine, and Associate Director, Internal Medicine Residency Program, NYU Langone Hospital-Brooklyn
| | - Rachael W Hayes
- Instructor, Department of Medicine, and Associate Director, Internal Medicine Residency Program, NYU Langone Hospital-Brooklyn
| | - Margaret Horlick
- Assistant Professor, Department of Medicine, and Senior Associate Director, Internal Medicine Residency Program, NYU Langone Health
| | - Jennifer G Adams
- Assistant Professor, Department of Medicine, and Co-Director, Primary Care Internal Medicine Residency Program, NYU Langone Health
| | - Sondra R Zabar
- Professor, Department of Medicine, Director, Division of General Internal Medicine and Clinical Innovation, and Director, Standardized Patient Program, NYU Langone Health
| |
Collapse
|
14
|
Wilhite JA, Fisher H, Altshuler L, Cannell E, Hardowar K, Hanley K, Gillespie C, Zabar S. Gasping for air: measuring patient education and activation skillsets in two clinical assessment contexts. BMJ SIMULATION & TECHNOLOGY ENHANCED LEARNING 2020; 7:428-430. [DOI: 10.1136/bmjstel-2020-000759] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/13/2020] [Revised: 11/06/2020] [Accepted: 11/17/2020] [Indexed: 11/04/2022]
Abstract
Objective structured clinical examinations (OSCEs) provide a controlled, simulated setting for competency assessments, while unannounced simulated patients (USPs) measure competency in situ or real-world settings. This exploratory study describes differences in primary care residents’ skills when caring for the same simulated patient case in OSCEs versus in a USP encounter. Data reported describe a group of residents (n=20) who were assessed following interaction with the same simulated patient case in two distinct settings: an OSCE and a USP visit at our safety-net clinic from 2009 to 2010. In both scenarios, the simulated patient presented as an asthmatic woman with limited understanding of illness management. Residents were rated through a behaviourally anchored checklist on visit completion. Summary scores (mean % well done) were calculated by domain and compared using paired sample t-tests. Residents performed significantly better with USPs on 7 of 10 items and in two of three aggregate assessment domains (p<0.05). OSCE structure may impede assessment of activation and treatment planning skills, which are better assessed in real-world settings. This exploration of outcomes from our two assessments using the same clinical case lays a foundation for future research on variation in situated performance. Using both assessments during residency will provide a more thorough understanding of learner competency.
Collapse
|
15
|
Zabar S, Hanley K, Wilhite JA, Altshuler L, Kalet A, Gillespie C. In the room where it happens: do physicians need feedback on their real-world communication skills? BMJ Qual Saf 2019; 29:182-184. [PMID: 31704892 DOI: 10.1136/bmjqs-2019-010384] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/29/2019] [Indexed: 11/03/2022]
Affiliation(s)
- Sondra Zabar
- Department of Medicine, New York University School of Medicine, New York City, New York, USA
| | - Kathleen Hanley
- Department of Medicine, New York University School of Medicine, New York City, New York, USA
| | - Jeffrey A Wilhite
- Department of Medicine, New York University School of Medicine, New York City, New York, USA
| | - Lisa Altshuler
- Department of Medicine, New York University School of Medicine, New York City, New York, USA
| | - Adina Kalet
- Department of Medicine, New York University School of Medicine, New York City, New York, USA
| | - Colleen Gillespie
- Department of Medicine, New York University School of Medicine, New York City, New York, USA
| |
Collapse
|
16
|
Igniting activation: Using unannounced standardized patients to measure patient activation in smoking cessation. Addict Behav Rep 2019; 9:100179. [PMID: 31193839 PMCID: PMC6544561 DOI: 10.1016/j.abrep.2019.100179] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2018] [Revised: 03/26/2019] [Accepted: 03/26/2019] [Indexed: 01/01/2023] Open
Abstract
Introduction Despite a decline, smoking rates have remained high, especially in communities with lower income, education, and limited insurance options. Evidence shows that physician-initiated counseling on smoking cessation is effective and saves lives, and that specific skills are needed to appropriately lead this type of patient-physician communication. Residency is a critical moment for future physicians and may be the optimal time to learn, practice, and refine this skillset. Unannounced Standardized Patients (USPs) have been found to be effective, incognito evaluators of resident practices. Methods This study introduced rigorously trained actors (USPs) into two urban, safety-net clinics to assess resident ability to engage, activate, and counsel a pre-contemplative smoker. A complementary chart review assessed appropriate documentation in the patient's electronic health record (EHR) and its relationship to counseling style and prescribing practices. Results Resident scores (% well done) on patient education and engagement were low (33% and 23%, respectively). Residents who coupled cessation advice with an open discussion style activated their patients more than those who solely advised cessation across all comparable measures. On EHR documentation, residents who accurately documented smoking history were more likely to directly advise their patient to quit smoking when compared to residents who did not document (t(97) = 2.828, p = .006, Cohen's D = 0.56). Conclusions Results highlight the need to reinforce training in patient-centered approaches including motivational interviewing, counseling, and shared decision-making. Future research should focus on the effects of smokers in pre-contemplation on physician counseling style and examine the relationship between medical training and provider communication to guide interventions. Counseling, documentation, and prescribing all vary when residents meet an Unannounced Standardized Patient (USP) smoker. Patients are activated to quit smoking when a provider couples cessation advice with an open discussion of pros and cons. Training in motivational interviewing and shared decision-making can enhance patient-provider cessation communication.
Collapse
|