Examining the Effects of Narrative Commentary on Evaluators’ Summative Assessments of Resident Performance

Anchor-based, end-of-shift ratings are commonly used to conduct performance assessments of resident physicians. These performance evaluations often include narrative assessments, such as solicited or “free-text” commentary. Although narrative commentary can help to create a more detailed and specific assessment of performance, there are limited data describing the effects of narrative commentary on the global assessment process. This single-group, observational study examined the effect of narrative comments on global performance assessments. A subgroup of the clinical competency committee, blinded to resident identity, assigned a single, consensus-based performance score (1–6) to each resident based solely on end-of-shift milestone scores. De-identified narrative comments from end-of-shift evaluations were then included and the process was repeated. We compared milestone-only scores to milestone plus narrative commentary scores using a nonparametric sign test. During the study period, 953 end-of-shift evaluations were submitted on 41 residents. Of these, 535 evaluations included free-text narrative comments. In 17 of the 41 observations, performance scores changed after the addition of narrative comments. In two cases, scores decreased with the addition of free-text commentary. In 15 cases, scores increased. The frequency of net positive change was significant ( p = .0023). The addition of narrative commentary to anchor-based ratings significantly influenced the global performance assessment of Emergency Medicine residents by a committee of educators. Descriptive commentary collected at the end of shift may inform more meaningful appraisal of a resident’s progress in a milestone-based paradigm. The authors recommend clinical training programs collect unstructured narrative impressions of residents’ performance from supervising faculty.

Satisfaction Academy: A Novel Residency Curriculum to Improve the Patient Experience in the Emergency Department

Introduction

Patient satisfaction is a key indicator of health care value and an increasingly important metric used to assess emergency physician performance and often reimbursement. To our knowledge, there is no standardized curriculum within emergency medicine (EM) residency programs that focuses on the patient experience in EM.

Methods

Our novel resident curriculum is an organized approach to enhancing patient-centered care by optimizing the patient experience. It spans the academic year, with key topics organized into a quarterly time line. Topics include physician courtesy and respect, pain management, discussion of diagnostic and therapeutic interventions, timely communication, and delivery of quality care. Each quarter has three components: introduction/didactics, an interactive workshop, and stories and reflection. The instructional methods used include didactic lectures, role-playing, and group reflection and storytelling.

Results

Of 44 participants, 54.5% completed a preintervention survey, and 45.5% completed a postintervention survey. The surveys consisted of 5-point Likert scales measuring degree of agreement with statements that reflected desired behaviors and/or attitudes. On the postintervention survey, participants gave scores indicating general agreement with desired behaviors including sitting at the bedside, acknowledging all persons in the room, and giving an anticipated disposition, as well as with feeling more knowledgeable about patient satisfaction.

Discussion

Our Satisfaction Academy has filled a significant gap related to enhancing the patient experience. This curriculum is generalizable to other EM residency programs, and the interactive peer-to-peer format is both engaging and customizable.