A multi-institutional study of the Family Conference Objective Structured Clinical Exam: a reliable assessment of professional communication

Teaching Patient-Related Communication to Surgical Residents in Brief Training Sessions

OBJECTIVE

Effective provider-patient communication has several benefits; however, few surgical residency programs have communication training and surgical residents have limited time for education. We developed a communication curriculum with limited didactics and emphasis on practice. Our objective was to evaluate whether this time-limited intervention led to changes in surgical resident communication skills.

DESIGN

A 4-module curriculum was implemented for surgical residents (PGY2-4). Each 30-minute module focused on specific communication micro-skills: empathy, concerns and expectations, chunking information and avoiding jargon, and teach-back. Modules included brief didactics, simulated patient interactions, feedback, and debriefing. Precurriculum, residents completed a 2-station objective structured clinical examination (OSCE) and a survey on communication confidence. Residents evaluated each module and postcurriculum, completed another 2-station OSCE, confidence survey, and overall curriculum evaluation. Using validated rating scales, OSCEs were scored by 2 independent raters.

SETTING

Tertiary care, academic center with a 5-year surgical residency program.

PARTICIPANTS

All 17 eligible residents completed both OSCEs and surveys, and 14 attended ≥3 modules.

RESULTS

Following the curriculum, residents reported increased use of the targeted skills and increased confidence in responding to emotions, information sharing, and bad news telling (p < 0.004). There was no change in history taking. Residents rated the usefulness of each module modestly (2.5-3.1, scale 0-4), however, the likelihood of skill implementation was higher (3.2-3.6). The overall postcurriculum OSCE scores increased (versus precurriculum scores, p < 0.001). Postcurriculum scores increased for empathy, concerns and expectations, and teach-back. Chunking information and avoiding jargon was unchanged. Fifteen residents reported module length as appropriate, and 2 thought they were too short.

CONCLUSIONS

The brief modules led to increased self-reported use of communication skills and were effective in improving resident communication in OSCEs. This may be a useful curricular model for both surgical and nonsurgical residency programs with limited availability for curricular time.

Palliative Care and Communication Training in Neurosurgery Residency: Results of a Trainee Survey

OBJECTIVE

Neurosurgeons care for critically ill patients near the end of life, yet little is known about how well their training prepares them for this role. We surveyed a random sample of neurosurgery residents to describe the quantity and quality of teaching activities related to serious illness communication and palliative care, and resident attitudes and perceived preparedness to care for seriously ill patients.

METHODS

A previously validated survey instrument was adapted to reflect required communication and palliative care competencies in the 2015 the Accreditation Council for Graduate Medical Education (ACGME) Milestones for Neurological Surgery. The survey was reviewed for content validity by independent faculty neurosurgeons, piloted with graduating neurosurgical residents, and distributed online in August 2016 to neurosurgery residents in the United States using the American Association of Neurological Surgeons (AANS)/Congress of Neurological Surgeons (CNS) Joint Section on Neurotrauma and Critical Care email listserv. Multiple choice and Likert scale responses were analyzed using descriptive statistics.

RESULTS

Sixty-two responses were recorded between August 2016 and October 2016. Most respondents reported no explicit teaching on: explaining risks and benefits of intubation and ventilation (69%), formulating prognoses in neurocritical care (60%), or leading family meetings (69%). Compared to performing craniotomies, respondents had less frequent practice leading discussions about withdrawing life-sustaining treatment (61% vs. 90%, p < 0.01, "weekly or more frequently"), and were less often observed (18% vs. 87%, p < 0.01) and given feedback on their performance (11% vs. 58%, p < 0.01). Nearly all respondents (95%) felt "prepared to discuss withdrawing life-sustaining treatments," however half (48%) reported they "would benefit from more communication training during residency." Most (87%) reported moral distress, agreeing that they "participated in operations and worried whether surgery aligned with patient goals."

CONCLUSIONS

Residents in our sample reported limited formal training, and relatively less observation and feedback, on required ACGME competencies in palliative care and communication. Most reported preparedness in this domain, but many were receptive to more training. Better quality and more consistent palliative care education in neurosurgery residency could improve competency and help ensure that neurosurgical care aligns with patient goals.

Communication Skills Training for General Surgery Residents

BACKGROUND

Although good communication skills are essential for surgeons, there is no formal communication training during general surgery residency.

OBJECTIVES

To implement a communication skills training program based on evidence-based teaching methods in general surgery residency.

DESIGN

We developed a 2-hour communication skills training program for general surgery residents, consisting of a small group skill practice session using role play with simulated patients along with real-time feedback from facilitators and observing peer residents. A board-certified palliative care physician and a board-certified surgeon facilitated each session. Outcome measures were self-assessment of preparedness with the session immediately before and after the session and 2 months after the session, resident satisfaction, and self-report frequency of skill practice. Results were compared between junior residents (postgraduate year 1-3) and senior residents (postgraduate year 4-5).

SETTING

Columbia University Medical Center in New York City, a tertiary care, urban academic center with a 5-year General Surgery Residency program.

PARTICIPANTS

Thirty-one out of 39 (79.4%) general surgery residents (20 junior and 11 senior) were trained over a 9-month period. All participants completed the immediate pre- and post-session surveys, and twenty residents (64.5%) completed the 2-month postsession follow-up survey.

RESULTS

Overall, self-assessment of preparedness for specific communication challenges improved significantly for 7 of 11 tasks. At baseline, senior residents felt significantly more prepared than junior residents in all 11 tasks. Junior residents' self-assessment of preparedness improved significantly in 10 of 11 tasks. Overall satisfaction with the session was very high (mean 4.74 on a 5-point scale). Residents reported high frequency of self-directed skill practice in the 2-month follow-up survey.

CONCLUSIONS

This 2-hour communication skills practice session for general surgery residents was feasible, and it improved resident self-assessment of preparedness in communication and augmented self-directed skill practice.

Tell Me Straight: Teaching Residents to Disclose Adverse Events in Surgery

OBJECTIVE

The purpose of this effort was to create an educational experience that provided learners a realistic disclosure experience and improved resident confidence discussing an adverse outcome with a patient and family.

DESIGN

Residents practiced disclosing a surgical complication to a patient/family with simulated patients (SPs). We paired professional SPs with former patient SPs to present a realistic case. Junior residents were given extra training time before their disclosure of a laparoscopic cholecystectomy conversion to an open procedure; senior residents disclosed a bile duct injury. Residents rated pre and post-module confidence levels, and skills performance using the Disclosure of a Complication Checklist.

SETTING

900-bed tertiary care hospital with surgical residency program and simulation center.

PARTICIPANTS

General surgery residents (PGY 1-5).

RESULTS

Eighteen residents participated in the disclosure module. Analysis of the medians and interquartile ranges of pre and post-module confidence scores showed significant improvement for each individual item and mean score of learners. Residents assessed their completion rates of individual Checklist tasks positively. For example, 94% self-endorsed completion of "explanation of facts," 89% self-endorsed "took responsibility," and 78% self-endorsed "apologized sincerely." Self-rated competence scores from the Checklist were low: 7% indicated they would be "extremely comfortable" entrusting their loved one's care to themselves, 11% rated their ability to explain the facts as "outstanding," and 12% felt they were "outstanding" in their "ability to disclose a complication in a professional manner."

CONCLUSION

Residents received important skills practice in our disclosure training; disclosure confidence increased after participation. Residents scored high on completion of disclosure tasks and low on comfort and proficiency of those tasks. The Checklist provided a useful set of tasks to review and complete in the exercise. Separating residents by PGY level enabled senior residents to experience a more complex scenario and junior residents extra time to practice.

Training Surgeons and Anesthesiologists to Facilitate End-of-Life Conversations With Patients and Families: A Systematic Review of Existing Educational Models

OBJECTIVE

Despite caring for patients near the end-of-life (EOL), surgeons and anesthesiologists report low confidence in their ability to facilitate EOL conversations. This discrepancy exists despite competency requirements and professional medical society recommendations. The objective of this systematic review is to identify articles describing EOL communication training available to surgeons and anesthesiologists, and to assess their methodological rigor to inform future curricular design and evaluation.

METHODS

This PRISMA-concordant systematic review identified English-language articles from PubMed, EMBASE, and manual review. Eligible articles included viewpoint pieces, and observational, qualitative, or case studies that featured an educational intervention for surgeons or anesthesiologists on EOL communication skills. Data on the study objective, setting, design, participants, intervention, and results were extracted and analyzed. The Newcastle-Ottawa Scale was used to assess methodological quality.

RESULTS

Database and manual search returned 2710 articles. A total of 2268 studies were screened by title and abstract, 46 reviewed in full-text, and 16 included in the final analysis. Fifteen studies were conducted exclusively in academic hospitals. Two studies included attending surgeons as participants; all others featured residents, fellows, or a mix thereof. Fifteen studies used simulated role-playing to teach and assess EOL communication skills. Measured outcomes included knowledge, attitudes, confidence, self-rated or observer-rated communication skills, and curriculum feedback; significance of results varied widely. Most studies lacked adequate methodological quality and appropriate control groups to be confident about the significance and applicability of their results.

CONCLUSIONS

There are few quality studies evaluating EOL communication training for surgeons and anesthesiologists. These programs frequently use role-playing to teach and assess EOL communication skills. More studies are needed to evaluate the effect of these interventions on patient outcomes. However, evaluating the effectiveness of these initiatives poses methodological challenges.

Tough Conversations: Development of a Curriculum for Medical Students to Lead Family Meetings

Background:

Few educational interventions have been developed to teach Family Meeting (FM) communication skills at the undergraduate level. We developed an innovative curriculum to address this gap.

Methods:

Fourth year medical students during 2011-2013 (n = 674) completed training for conducting a FM. To assess the effectiveness of this training, students completed a FM Objective Structured Clinical Exam (OSCE) that included 15 domains rated on a 1-5 point Likert scale. Tasks included discussing prognosis, establishing goals of care and demonstrating conflict resolution skills. Students received one-to-one feedback from standardized family members and faculty observers. Group debriefings with faculty were held after the OSCE.

Results:

Analysis of faculty feedback narratives revealed four themes in which students required improvement: 1) Discussing prognosis, 2) Explaining palliative care/hospice, 3) Avoiding medical jargon, and 4) Discussing cultural/religious preferences. Evaluation total mean score was 28.2 (Min 15, Max 63; SD 7.57), and identified student’s need to; 1) Ask more about the degree of knowledge family members want, 2) Ask religious beliefs, and 3) Assess family members’ level of education ( p < 0.001). Qualitative analysis of group debriefings suggested that student perception of the OSCE experience was positive overall. Students found the case to be realistic and immediate feedback to be helpful.

Conclusions:

Conducting a FM is an advanced skill. This study shows that it is possible to train fourth year students to lead FMs and identify their strengths, needs using a FM OSCE.

An objective structured clinical exam to measure intrinsic CanMEDS roles

BACKGROUND

The CanMEDS roles provide a comprehensive framework to organize competency-based curricula; however, there is a challenge in finding feasible, valid, and reliable assessment methods to measure intrinsic roles such as Communicator and Collaborator. The objective structured clinical exam (OSCE) is more commonly used in postgraduate medical education for the assessment of clinical skills beyond medical expertise.

METHOD

We developed the CanMEDS In-Training Exam (CITE), a six-station OSCE designed to assess two different CanMEDS roles (one primary and one secondary) and general communication skills at each station. Correlation coefficients were computed for CanMEDS roles within and between stations, and for general communication, global rating, and total scores. One-way analysis of variance (ANOVA) was used to investigate differences between year of residency, sex, and the type of residency program.

RESULTS

In total, 63 residents participated in the CITE; 40 residents (63%) were from internal medicine programs, whereas the remaining 23 (37%) were pursuing other specialties. There was satisfactory internal consistency for all stations, and the total scores of the stations were strongly correlated with the global scores r=0.86, p<0.05. Noninternal medicine residents scored higher in terms of the Professional competency overall, whereas internal medicine residents scored significantly higher in the Collaborator competency overall.

DISCUSSION

The OSCE checklists developed for the assessment of intrinsic CanMEDS roles were functional, but the specific items within stations required more uniformity to be used between stations. More generic types of checklists may also improve correlations across stations.

CONCLUSION

An OSCE measuring intrinsic competence is feasible; however, further development of our cases and checklists is needed. We provide a model of how to develop an OSCE to measure intrinsic CanMEDS roles that educators may adopt as residency programs move into competency-based medical education.

Teaching professionalism in graduate medical education: What is the role of simulation?

BACKGROUND

We systematically reviewed the literature concerning simulation-based teaching and assessment of the Accreditation Council for Graduate Medical Education professionalism competencies to elucidate best practices and facilitate further research.

METHODS

A systematic review of English literature for "professionalism" and "simulation(s)" yielded 697 abstracts. Two independent raters chose abstracts that (1) focused on graduate medical education, (2) described the simulation method, and (3) used simulation to train or assess professionalism. Fifty abstracts met the criteria, and seven were excluded for lack of relevant information. The raters, 6 professionals with medical education, simulation, and clinical experience, discussed 5 of these articles as a group; they calibrated coding and applied further refinements, resulting in a final, iteratively developed evaluation form. The raters then divided into 2 teams to read and assess the remaining articles. Overall, 15 articles were eliminated, and 28 articles underwent final analysis.

RESULTS

Papers addressed a heterogeneous range of professionalism content via multiple methods. Common specialties represented were surgery (46.4%), pediatrics (17.9%), and emergency medicine (14.3%). Sixteen articles (57%) referenced a professionalism framework; 14 (50%) incorporated an assessment tool; and 17 (60.7%) reported debriefing participants, though in limited detail. Twenty-three (82.1%) articles evaluated programs, mostly using subjective trainee reports.

CONCLUSION

Despite early innovation, reporting of simulation-based professionalism training and assessment is nonstandardized in methods and terminology and lacks the details required for replication. We offer minimum standards for reporting of future professionalism-focused simulation training and assessment as well as a basic framework for better mapping proper simulation methods to the targeted domain of professionalism.

Unanticipated teaching demands rise with simulation training: strategies for managing faculty workload

INTRODUCTION

Using simulation to teach and assess learners represents a powerful approach to training, but one that comes with hidden costs in terms of faculty time, even if programs adopt existing curricula. Some simulators are built to be used independently by learners, but much of the surgical simulation curricula developed for cognitive and psychomotor tasks requires active faculty involvement and low learner-to-faculty teaching ratios to ensure sufficient practice with feedback. The authors hypothesize that the added teaching demands related to simulation have resulted in a significant financial burden to surgery training programs. To date, the effect of simulation-based training on faculty workload has not been estimated objectively and reported in the literature.

METHODS

To test their hypothesis, the authors analyzed data from 2 sources: (1) changes over time (2006-2014) in formal teaching hours and estimated faculty costs at the University of Minnesota, General Surgery Department and (2) a 2014 online survey of general surgery program directors on their use of simulation for teaching and assessment and their perceptions of workload effects.

RESULTS

At the University of Minnesota, the total number of hours spent by department faculty in resident and student simulation events increased from 81 in annual year 2006 to 365 in annual year 2013. Estimated full-time equivalent faculty costs rose by 350% during the same period. Program directors (n = 48) of Association of Program Directors in Surgery reported either a slight (60%) or a significant (33%) increase in faculty workload with the advent of simulation, and moderate difficulty in finding enough instructors to meet this increase. Calling upon leadership for support, using diverse instructor types, and relying on "the dedicated few" represent the most common strategies.

CONCLUSION

To avoid faculty burnout and successfully sustain faculty investment in simulation-based training over time, programs need to be creative in building, sustaining, and managing the instructor workforce.

Colorectal surgeons teaching general surgery residents: current challenges and opportunities

Effective teaching for general surgery residents requires that faculty members with colorectal expertise actively engage in the education process and fully understand the current context for residency training. In this article, we review important national developments with respect to graduate medical education that impact resident supervision, curriculum implementation, resident assessment, and program evaluation. We argue that establishing a culture of respect and professionalism in today's teaching environment is one of the most important legacies that surgical educators can leave for the coming generation. Faculty role modeling and the process of socializing residents is highlighted. We review the American College of Surgeons' Code of Professional Conduct, summarize some of the current strategies for teaching and assessing professionalism, and reflect on principles of motivation that apply to resident training both for the trainee and the trainer.

Are pediatric critical care medicine fellowships teaching and evaluating communication and professionalism?

OBJECTIVES

To describe the teaching and evaluation modalities used by pediatric critical care medicine training programs in the areas of professionalism and communication.

DESIGN

Cross-sectional national survey.

SETTING

Pediatric critical care medicine fellowship programs.

SUBJECTS

Pediatric critical care medicine program directors.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Survey response rate was 67% of program directors in the United States, representing educators for 73% of current pediatric critical care medicine fellows. Respondents had a median of 4 years experience, with a median of seven fellows and 12 teaching faculty in their program. Faculty role modeling or direct observation with feedback were the most common modalities used to teach communication. However, six of the eight (75%) required elements of communication evaluated were not specifically taught by all programs. Faculty role modeling was the most commonly used technique to teach professionalism in 44% of the content areas evaluated, and didactics was the technique used in 44% of other professionalism content areas. Thirteen of the 16 required elements of professionalism (81%) were not taught by all programs. Evaluations by members of the healthcare team were used for assessment for both competencies. The use of a specific teaching technique was not related to program size, program director experience, or training in medical education.

CONCLUSIONS

A wide range of techniques are currently used within pediatric critical care medicine to teach communication and professionalism, but there are a number of required elements that are not specifically taught by fellowship programs. These areas of deficiency represent opportunities for future investigation and improved education in the important competencies of communication and professionalism.

Assessing professionalism: summary of the working group on assessment of observable learner performance

Professionalism is one of the six Accreditation Council on Graduate Medical Education (ACGME) core competencies on which emergency medicine (EM) residents are assessed. However, very few assessment tools exist that have been rigorously evaluated in this population. One goal of the 2012 Academic Emergency Medicine consensus conference on education research in EM was to develop a research agenda for testing and developing tools to assess professionalism in EM residents. A literature review was performed to identify existing assessment tools. Recommendations on future research directions were presented at the consensus conference, and an agenda was developed.

Simulation in surgical education

The pedagogical approach to surgical training has changed significantly over the past few decades. No longer are surgical skills solely acquired through a traditional apprenticeship model of training. The acquisition of many technical and nontechnical skills is moving from the operating room to the surgical skills laboratory through the use of simulation. Many platforms exist for the learning and assessment of surgical skills. In this article, the authors provide a broad overview of some of the currently available surgical simulation modalities including bench-top models, laparoscopic simulators, simulation for new surgical technologies, and simulation for nontechnical surgical skills.

Integrating palliative care in the surgical and trauma intensive care unit: a report from the Improving Palliative Care in the Intensive Care Unit (IPAL-ICU) Project Advisory Board and the Center to Advance Palliative Care

OBJECTIVE

Although successful models for palliative care delivery and quality improvement in the intensive care unit have been described, their applicability in surgical intensive care unit settings has not been fully addressed. We undertook to define specific challenges, strategies, and solutions for integration of palliative care in the surgical intensive care unit.

DATA SOURCES

We searched the MEDLINE database from inception to May 2011 for all English language articles using the term "surgical palliative care" or the terms "surgical critical care," "surgical ICU," "surgeon," "trauma" or "transplant," and "palliative care" or "end-of- life care" and hand-searched our personal files for additional articles. Based on review of these articles and the experiences of our interdisciplinary expert Advisory Board, we prepared this report.

DATA EXTRACTION AND SYNTHESIS

We critically reviewed the existing literature on delivery of palliative care in the surgical intensive care unit setting focusing on challenges, strategies, models, and interventions to promote effective integration of palliative care for patients receiving surgical critical care and their families.

CONCLUSIONS

Characteristics of patients with surgical disease and practices, attitudes, and interactions of different disciplines on the surgical critical care team present distinctive issues for intensive care unit palliative care integration and improvement. Physicians, nurses, and other team members in surgery, critical care and palliative care (if available) should be engaged collaboratively to identify challenges and develop strategies. "Consultative," "integrative," and combined models can be used to improve intensive care unit palliative care, although optimal use of trigger criteria for palliative care consultation has not yet been demonstrated. Important components of an improvement effort include attention to efficient work systems and practical tools and to attitudinal factors and "culture" in the unit and institution. Approaches that emphasize delivery of palliative care together with surgical critical care hold promise to better integrate palliative care into the surgical intensive care unit.

Pilot evaluation of a biopsychosocial integrated standardized patient examination in a family medicine clerkship

OBJECTIVE

A new biopsychosocial Integrated Standardized Patient Examination (ISPE) was implemented to assess communication skills for medical students participating in a family medicine clerkship.

METHOD

Mixed method, multi-level evaluation.

RESULTS

Pilot ISPE scores were significantly higher than previous Objective Structured Clinical Examination (OSCE) (p < .01). Family Physician and Social/Behavioral Scientist rater scores were correlated (p < .001), and Cronbach's alpha statistics were acceptable (FM: alpha = .837; BH: alpha = .768). Preceptor scores on "relations with professionals" (beta = .694, p = .008) significantly predicted ISPE scores, but other clerkship assignment grades were inversely associated with ISPE. Qualitative focus group themes included lack of readiness, divergence in beliefs on scope of physician practice, and focus on grade.

CONCLUSIONS

This pilot project describes the process and demonstrates the possibility of implementing a complex standardized patient case to assess students' management of complicated primary care patients with medical, psychological, and social issues.