Putting the Heme Back in Heme-Onc: an Evaluation of the Redesigned Hematology Track within the Dana Farber/Mass General Brigham Hematology-Oncology Fellowship

In 2019, the Dana Farber/Mass General Brigham Hematology-Oncology Fellowship redesigned the 6-month Hematology training track/curriculum required for all fellows seeking to double board in hematology and oncology. Responding to both national and local trends suggesting a future shortage of hematologists, the goal of the redesign was to create a new curriculum that would increase fellow interest in hematology, improve fellows' clinical knowledge of hematology, and serve as an example to other Hematology-Oncology programs across the country. The revised track has now been in place for four years, and, in this paper, the authors present the fellow experience with the first four years of the redesigned curriculum. Based on the number of fellows who chose to complete the new curriculum, as well as the fellow evaluations and performance on the Hematology In-Training Exam, the authors conclude that the new curriculum has successfully increased both fellow interest in and knowledge of hematology.

Nurse-doctor co-teaching: A path towards interprofessional collaboration

BACKGROUND

Nurse-doctor collaborations are essential for team-based patient care. Although there are increasing calls for interprofessional education, teaching and learning together is rare. In 2019, we designed a Nurse-Doctor Co-Teaching pilot programme to provide an opportunity for nurses and doctors to co-teach junior doctors and nurses. We aimed to explore the experiences of the co-teachers and understand their perceptions of teaching together. The study was conducted through the lens of positioning theory.

METHODS

We held an hour-long focus group discussion and follow-up one-on-one interviews with nurses and doctors who participated as co-teachers. Conversations were audio-video recorded, transcribed, and thematically analysed. The Partners Institutional Review Board approved this study.

RESULTS

Three nurses and four doctors participated in the focus group conversation, and four nurses and two doctors participated in individual interviews. Participant narratives provided insight into shifts in hospital culture that would be necessary to promote effective interprofessional learning and collaboration: (1) break down professional silos, (2) invite the nursing perspective, (3) flatten professional hierarchies, and (4) recognise nurses as clinical teachers.

CONCLUSION

Nurses and doctors felt they shared a collegial and equal partnership as co-teachers. But this relationship was not typical of their daily clinical roles. Institutional barriers presented challenges to collaboration on the hospital floor and nursing participation in teaching. Successful interprofessional education may require culture and policy shifts that formally recognise nurses as valuable clinical teachers.

The Harvard Medical School Pathways curriculum: A comprehensive curricular evaluation

PURPOSE

The Harvard Medical School Pathways curriculum represents a major reform effort. Our goals were to enhance reasoning and clinical skills and improve the learning environment and students' approach to learning via use of collaborative, case-based pedagogy; early clinical exposure; and enhanced approaches to teaching and evaluating clinical skills. We evaluated the impact of Pathways on key outcomes related to these goals.

MATERIALS AND METHODS

In this prospective, mixed-methods study, we compared the last prior-curriculum cohort (2014 matriculation, n = 135) and first new-curriculum cohort (2015 matriculation, n = 135). Measures included Likert-type surveys, focus groups, and test scores to assess outcomes.

RESULTS

Compared with prior-curriculum students, new-curriculum students reported higher mean preclerkship learning environment ratings (Educational Climate Inventory, 62.4 versus 51.9, p < 0.0001) and greater satisfaction with the quality of their preclerkship education (88% versus 73%, p = 0.0007). Mean USMLE Step-1 and Step-2 scores did not differ between groups. At graduation, new-curriculum students rated their medical school experience higher in 6 of 7 domains, including 'fostering a culture of curiosity and inquiry' (4.3 versus 3.9, p = 0.006) and focus on 'student-centered learning' (3.9 versus 3.4, p = 0.002).

CONCLUSIONS

The new curriculum outperformed or was equal to the prior one on most measures of learning environment and perceived quality of education, without a decline in medical knowledge or clinical skills. Robust longitudinal evaluation provided important feedback for ongoing curriculum improvement.

Impact of Extended Duty Hours on Perceptions of Care and Objective Patient Outcomes

OBJECTIVE

In 2017, interns were permitted to work continuously for up to 28 hours at a time, a reversal from the previously mandated 16-hour limit. Our objective was to evaluate perceptions of care and patient outcomes on an extended (28-hour) compared with a limited (16-hour) duty-hour system on identical interdisciplinary teams.

METHODS

Sixty-two interns, 27 residents, 28 attendings, and 449 patients participated. Patients completed surveys assessing their satisfaction. Anonymous weekly surveys were obtained from interns, residents, and attendings evaluating perceptions of intern tiredness, overall satisfaction, and performance. Nursing surveys evaluated intern and medical team performance. Objective outcome measures, including intensive care unit transfers, length of stay, readmissions, mortality, and complications, were assessed through a retrospective, blinded chart review.

RESULTS

Patients reported similar satisfaction in care. Extended duty-hour interns reported significantly decreased familiarity with their patients, decreased ability to conduct physical exams on new patients, increased tiredness, and decreased overall satisfaction. Residents overseeing extended-duty interns reported significantly decreased quality in intern presentations and overall quality of teaching, and increased perception of intern tiredness and increased incorrect orders. Attending physicians reported significantly improved quality of new patient presentations by extended duty-hour interns. No significant differences in patient objective outcome measures were noted.

CONCLUSIONS

Extended intern duty hours do not affect patient's satisfaction with their care. Although interns in the extended duty-hour system reported significantly increased fatigue and decreased overall satisfaction and residents' perceived increases in incorrect intern orders in the extended duty-hour system, there were no detrimental effects on patient safety.

Agreement of Nurses' and Physicians' Attitudes on Collaboration During the Covid-19 Pandemic Using the Jefferson Scale of Attitudes Toward Physician-Nurse Collaboration

BACKGROUND

Collaboration between physicians and nurses has been shown to lead to better patient outcomes. However, studies have shown differing physicians' and nurses' responses to survey questions about physician-nurse collaboration. We surveyed physicians and nurses during the Covid-19 pandemic for their attitudes toward collaboration.

METHODS

In August 2021, during the Covid-19 pandemic, we surveyed physicians and nurses throughout an urban, academic teaching hospital over a consecutive twenty-day period using the validated Jefferson Scale of Attitudes Toward Physician-Nurse Collaboration. Anonymous surveys were obtained from nurses and physicians on duty at the hospital. Demographic data from each survey included gender, age, profession of nurse or physician, degree, and specialization.

RESULTS

Four hundred and fifteen (415) unique paper surveys were collected from 308 nurses and 107 physicians over the twenty-day period. Five nurses and two physicians declined to complete the survey (1.6%). Using the Independent t-test of Means, total score and sub-scores were analyzed. Physicians and nurses scored the paper surveys in a similar manner. No statistically significant differences between the scores of physicians and nurses were found for any of the fifteen Jefferson Scale of Attitudes Toward Physician-Nurse Collaboration questions, except for the youngest age group (20-29-year-old) having a significantly more positive response to doctors being the dominant authority on all health matters (p-value=0.011). Gender and nursing degree did not make a significant difference. Surgical Specialties (167), Medical Specialties (196), Intensive Care Unit (21), and the Emergency Department (43) survey responses did not differ significantly from each other.

CONCLUSION

One and a half years into the Covid-19 pandemic, physicians and nurses at an urban, academic teaching hospital were in agreement with their responses on the validated Jefferson Scale of Attitudes Toward Physician-Nurse Collaboration. Our data may reflect a catalytic and positive effect of the Covid-19 pandemic on physician and nurse attitudes toward collaboration.

Preparing Future Doctors for Telemedicine: An Asynchronous Curriculum for Medical Students Implemented During the COVID-19 Pandemic

PROBLEM

The COVID-19 pandemic led to changes in both the clinical environment and medical education. The abrupt shift to telemedicine in March 2020, coupled with the recommendation that medical students pause in-person clinical rotations, highlighted the need for student training in telemedicine.

APPROACH

To maintain students' ability to participate in clinical encounters and continue learning in the new virtual environment, a telemedicine curriculum for clinical students was rapidly developed at Harvard Medical School (HMS) focusing on the knowledge and skills needed to conduct live video encounters. Curriculum leads created an interactive, flexible curriculum to teach students clinical skills, regulatory issues, professionalism, and innovations in telemedicine. This 5-module curriculum was delivered using various primarily asynchronous modalities including webinar-style presentations, prerecorded videos of physical exams from different disciplines, shadowing a synchronous telemedicine visit, peer discussions in small groups, and quizzes with both multiple-choice and open-ended questions.

OUTCOMES

During May 2020, 252 clerkship and postclerkship medical students at HMS completed the telemedicine curriculum. All students completed a precourse survey and 216 (85.7%) completed the postcourse survey. Students' self-rated knowledge of telemedicine increased, on average, from 38 (15.1%) reporting being fairly/very knowledgeable over 4 domains before the course to 182 (84.3%) afterward (P < .001). The course was highly rated, with 176/205 (85.9%) students reporting that it met their learning needs and 167/205 (81.5%) finding the delivery methods to be effective. Of 101 (45.3%) students who answered an open-ended postcourse survey question, 91 (90.1%) reported asynchronous learning to be a positive experience.

NEXT STEPS

As telemedicine becomes increasingly and likely permanently integrated into the health care system, providing medical students with robust training in conducting care virtually will be essential. This curriculum provides a promising and feasible framework upon which other schools can apply these emerging competencies to design their own telemedicine curricula.

Nurse-Doctor Co-Teaching: A Pilot Study of the Design, Development, and Implementation of Structured Interprofessional Co-Teaching Sessions

INTRODUCTION

High levels of interprofessional collaboration are beneficial for patients and healthcare providers. Co-teaching may be one method for creating a collaborative environment. This pilot study designed, developed, and implemented Nurse-Doctor Co-Teaching on an inpatient medicine service.

METHODS

Ten Nurse-Doctor Co-Teaching pairs designed 30-minute, structured co-teaching sessions with learning objectives, evidence-based content, interactive teaching strategies and a Take-Away of key content with the help of a coaching team. Each session was presented by a nurse and senior doctor to nurse and resident learners. Our assessment blueprint included: 1. Anonymous surveys assessing the overall rating of each session and 2. Pre- and post-anonymous surveys assessing measures of interprofessional collaboration and communication between nurses and residents before and after the series of ten co-teaching sessions.

RESULTS

Data from ten post-session surveys included 121 of 156 participants (77.6%). Attendance at each session ranged from 13-19 participants with 8-17 participants completing a survey per session for an average of 12.1 surveys analyzed. All Nurse-Doctor Co-Teaching sessions scored in the excellent range between 1.00 and 1.43 on a Likert scale (1 is excellent and 5 is poor). In response to the question "What did you like best?", interactive teaching strategies was the most frequent spontaneous answer. A significant correlation between the number of interactive teaching strategies and enjoyability of the session (p-value=0.01) was observed. Measures of interprofessional collaboration and communication did not change significantly in the pre-intervention compared to post-intervention period.

CONCLUSION

We created a unique model of interprofessional co-teaching on an inpatient service. The overall excellent ratings of our interactive sessions indicate that Nurse-Doctor Co-Teaching is a valued form of learning. Our structured format is adaptable to various medical settings and could be expanded to include additional allied health professionals. We plan further studies to assess if Nurse-Doctor Co-Teaching improves measures of interprofessional collaboration.

Optimizing Multidisciplinary Simulation in Medical School for Larger Groups: Role Assignment by Lottery and Guided Learning

PURPOSE

Medical school simulations are often designed for a limited number of students to maximize engagement and learning. To ensure that all first-year medical students who wished to join had an opportunity to participate, we designed a novel method for larger groups.

PATIENTS AND METHODS

We devised a low technology "Orchestra Leader's" chart approach to prominently display students' roles, chosen by lottery. During simulation, the chart was mounted on an intravenous pole and served as a group organizational tool. A course instructor prompted students using the chart to accomplish the course objectives in a logical order. Real-life cardiologists and gastroenterologists provided the students with expert subspecialty consultation. We analyzed 125 anonymous student evaluation ratings for 3 years (2017-2019) with a range of 8 to 19 students per laboratory session.

RESULTS

Our 2017-2019 larger group sessions were all rated as excellent (1.26, Mean, SD ±.510) on the Likert scale where 1.0 is excellent and 5.0 is poor. There were no statistically significant differences in overall ratings among the 2017, 2018 and 2019 sessions. The subspecialists were uniformly rated as excellent. Verbatim free-text responses demonstrated resounding student appreciation for the role assignment by lottery method.

CONCLUSION

We designed a novel, "Orchestra Leader's" chart approach for accommodating larger groups in a multidisciplinary simulation laboratory using role assignment by lottery, roles depicted on an organizational chart, and expert instructor prompting. Our consistently excellent ratings suggest that our methods are useful for achieving well-rated larger group simulation laboratories.

Is Asking Questions on Rounds a Teachable Skill? A Randomized Controlled Trial to Increase Attendings' Asking Questions

BACKGROUND

Morning bedside rounds remain an essential part of Internal Medicine residency education, but rounds vary widely in terms of educational value and learner engagement.

OBJECTIVE

To evaluate the efficacy of an intervention to increase the number and variety of questions asked by attendings at the bedside and assess its impact.

DESIGN

We conducted a randomized, controlled trial to evaluate the efficacy of our intervention.

PARTICIPANTS

Hospitalist attendings on the general medicine service were invited to participate. Twelve hospitalists were randomized to the experimental group and ten hospitalists to the control group.

INTERVENTION

A one-hour interactive session which teaches and models the method of asking questions using a non-medical case, followed by practice using role plays with medical cases.

MAIN MEASURES

Our primary outcome was the number of questions asked by attendings during rounds. We used audio-video recordings of rounds evaluated by blinded reviewers to quantify the number of questions asked, and we also recorded the type of question and the person asked. We assessed whether learners found rounds worthwhile using anonymous surveys of residents, patients, and nurses.

KEY RESULTS

Blinded analysis of the audio-video recordings demonstrated significantly more questions asked by attendings in the experimental group compared to the control group (mean number of questions 23.5 versus 10.8, p< 0.001) with significantly more questions asked of the residents (p<0.003). Residents rated morning bedside rounds with the experimental attendings as significantly more worthwhile compared to rounds with the control group attendings (p=0.009).

CONCLUSION

Our study findings highlight the benefits of a one-hour intervention to teach faculty a method of asking questions during bedside rounds. This educational strategy had the positive outcome of including significantly more resident voices at the bedside. Residents who rounded with attendings in the experimental group were more likely to "strongly agree" that bedside rounds were "worthwhile".

Do Professionalism Lapses in Medical School Predict Problems in Residency and Clinical Practice?

PURPOSE

Recognizing that physicians must exhibit high levels of professionalism, researchers have attempted to identify the precursors of clinicians' professionalism difficulties, typically using retrospective designs that trace sanctioned physicians back to medical school. To better establish relative risk for professionalism lapses in practice, however, this relationship must also be studied prospectively. Therefore, this study investigated the sequelae of medical school professionalism lapses by following students with medical school professionalism problems into residency and practice.

METHOD

Beginning in 2014, 108 graduates from Harvard Medical School and Case Western Reserve University School of Medicine who appeared before their schools' review boards between 1993 and 2007 for professionalism-related reasons were identified, as well as 216 controls matched by sex, minority status, and graduation year. Prematriculation information and medical school performance data were collected for both groups. Outcomes for the groups were studied at 2 points in time: ratings by residency directors, and state medical board sanctions and malpractice suits during clinical practice.

RESULTS

Compared with controls, students who appeared before their schools' review boards were over 5 times more likely to undergo disciplinary review during residency (16% vs 3%, respectively) and almost 4 times more likely to require remediation or counseling (35% vs 9%, respectively). During clinical practice, 10% of those who had made review board appearances were sued or sanctioned vs 5% of controls. Logistic regression for these outcomes indicated, however, that professional lapses in medical school were not the only, or even the most important, predictor of problems in practice.

CONCLUSIONS

Students with professionalism lapses in medical school are significantly more likely to experience professionalism-related problems during residency and practice, although other factors may also play an important predictive role.

Case-Based Curriculum With Integrated Smartphone Applications Improves Internal Medicine Resident Knowledge Of Contraceptive Care

BACKGROUND

Contraception is an essential preventive service for all women. However, the literature suggests that internal medicine residents have low levels of confidence and knowledge of contraceptive care.

OBJECTIVE

We designed and implemented a novel contraception curriculum promoting active, collaborative learning. We sought to evaluate whether this curriculum improved internal medicine resident knowledge of and comfort with contraceptive care through the administration of pre-/post-surveys.

METHODS

Our curriculum was delivered in a two-hour session as part of the mandatory ambulatory curriculum for internal medicine interns at our institution in the academic year 2017-2018. Interns were provided with select online resources and two smartphone applications at the beginning of the session, which they then used in case-based small group work. Small group work was followed by a large group case review, co-facilitated by OB/GYN and internal medicine faculty.

RESULTS

Thirty-eight participants completed surveys assessing knowledge of and comfort with contraceptive care immediately before and after the curriculum; 20 participants completed surveys assessing the same domains 4-6 months after the curriculum. Data from surveys administered immediately post-curriculum demonstrated significant improvements in knowledge about and comfort with counseling about, assessing medical eligibility for, and initiating multiple forms of contraception. Many of these improvements in knowledge and comfort were maintained on follow-up surveys 4-6 months following the curriculum.

CONCLUSION

Our case-based curriculum with integrated smartphone applications resulted in significant improvements in internal medicine resident knowledge of and comfort with the key skills of contraceptive care. In contrast to active, collaborative learning methodologies such as the flipped classroom, our methodology supports active, collaborative learning without requiring advance learner preparation, and is thus well suited to the time constraints of the graduate medical education setting. Our methodology is readily translatable to other clinical topics and residency curricula.

Teaching methods used by internal medicine residents on rounds: what works?

BACKGROUND

The specific teaching methods used by internal medicine residents on walk rounds are unknown.

OBJECTIVES

1) To characterize in real time the specific teaching methods used by internal medicine residents on rounds and 2) to identify attributes of successful resident teaching on rounds.

MATERIALS AND METHODS

We conducted a prospective mixed-methods study on inpatient medical services at a single academic medical center from September 2016 to January 2017. Participants were internal medicine residents (postgraduate year [PGY]-1, PGY-2, and PGY-3) and attending physicians. Teachers were PGY-2 and PGY-3 residents, and learners were PGY-1 residents. Residents' teaching on rounds was observed and characterized according to resident demographics, specific teaching methods, and length of time. Participants completed a survey with Likert scale and free-text questions.

RESULTS AND CONCLUSION

Among 136 surveys across 28 separate teaching encounters, we noted that PGY-3 residents' teaching was rated significantly better than PGY-2 residents' teaching. Teaching lasting >1 minute was rated significantly better than teaching lasting <1 minute. Free-text responses emphasized the value of immediate clinical relevance, citing published evidence, conciseness, clarity, and pertinence to the patient. Our findings may help guide internal medicine residents aiming to teach better on rounds and inform further research into specific resident teaching methods.

Asking what do residents value most: a recent overview of internal medicine residents' learning preferences

BACKGROUND

Little is known about the preferred learning experiences of today's internal medicine residents. We conducted a survey of the educational experiences in an internal medicine residency to determine the learning opportunities internal medicine residents value most and why.

METHODS

An online, anonymous survey of 182 internal medicine residents was performed, with each resident receiving a survey each day over nine days. Participants were asked to state their most valuable learning experience over the past day, describe why it was valuable, and rank it on a 5-point Likert-type scale. Resident free-text responses were coded and grouped into themes. The location of and participants in the experience were also examined.

RESULTS

The 182 residents completed a total of 303 surveys. Of the 303 surveys, 92% (N=277) of the responses noted their chosen learning experience was useful. An attending was involved in 50% (N=152) of experiences; the patient was noted as a participant in 8% (N=25) of experiences. Free-text responses were coded into five thematic groups descriptive of why residents found their learning experiences to be valuable: Repetition in Learning, Effective Pedagogy, Clinical Problem Solving as an Individual or Collaboratively, Opportunity for Active Engagement, and Bedside Learning.

CONCLUSION

Our data provide a broader framework for designing and implementing future faculty development and resident curricula that emphasize interprofessional education and the patient as a key educational figure.

Volunteer patients and small groups contribute to abdominal examination's success

BACKGROUND

Prior to 2007, we taught the abdominal examination in a hospital based group to 40 students, at one hospital. We used volunteer patients, small groups, repetition, and required faculty development sessions. In 2007, our medical school changed its "Introduction to Physical Examination" session so that the entire class was to be taught in a geographically central session. Our hospital was selected to lead the abdominal examination portion of the session.

AIM

Our aim was to answer three questions. First, could we quadruple the recruitment of volunteer patients, and faculty? Second, was it volunteer patients, small groups, repetition, or faculty training that was most valued by the students? Third, would volunteer patients and/or faculty agree to participate a second time?

METHODS

A total of 43-46 patients and 43-46 faculty were recruited and 43-46 examining rooms were obtained for each of the 5 years of this study. Teachers were required to attend a 1-hour faculty development session. The class of about 170 students was divided into 43-46 groups each year. The teacher demonstrated the abdominal examination and each student practiced the examination on another student. Each student then repeated the full abdominal examination on a volunteer patient.

RESULTS

Over the 5-year time period (2008-2012), the abdominal examination ranked first among all organ systems' "Introductory Sessions". The abdominal examination ratings had the best mean score (1.35) on a Likert scale where 1 is excellent and 5 is poor. The students gave the most positive spontaneous comments to having volunteer patients, with small groups coming in as the second most appreciated educational element.

CONCLUSION

We successfully quadrupled the number of faculty, patients, and examining rooms and created a highly rated educational program as measured by anonymous student evaluations, patient and faculty participation, and the medical school's selecting the abdominal examination methods as an "Advanced Examination" for the Pathways Curriculum.

The Effect of Rubric-Guided, Focused, Personalized Coaching Sessions and Video-Recorded Presentations on Teaching Skills Among Fourth-Year Medical Students: A Pilot Study

PROBLEM

As medical students become residents, teaching becomes an expected and integral responsibility. Yet, training-for-teaching opportunities are lacking. In 2014, the authors designed a pilot study using rubric-guided, focused, personalized coaching sessions and video-recorded presentations to improve student teaching skills among fourth-year students at Harvard Medical School.

APPROACH

In 2014-2015, the authors recruited students from an elective on how to tutor preclinical students for the pilot, which consisted of four phases: a precoaching teaching presentation, a 30- to 45-minute coaching session, a postcoaching teaching presentation, and blinded reviewer ratings. Students' pre- and postcoaching presentations were video recorded. Using a scoring rubric for 15 teaching skills, students rated their pre- and postcoaching videos. Blinded reviewers also rated the pre- and postcoaching presentations using the same rubric with an additional category to gauge their overall impression.

OUTCOMES

Fourteen students completed all four phases of the pilot. Students' ratings demonstrated statistically significant improvement in several teaching skills, including presentation content (P < .001), rate of speech (P = .001), and opening statement and learning objectives (P = .004). Blinded reviewers' ratings demonstrated statistically significant improvements in several teaching skills, including opening statement and learning objectives (P < .001), overall impression (P = .001), and conclusion and summary of learning objectives (P = .004). Students provided largely positive comments on the interventions.

NEXT STEPS

The authors will work toward addressing limitations in the rubric, using coaching in different teaching settings, addressing the interventions' generalizability, training coaches, and performing additional evaluations.

Innovative curriculum for second-year Harvard-MIT medical students: practicing communication skills with volunteer patients giving immediate feedback

PURPOSE

Medical students are expected to develop excellent communication skills. The purpose of our study was to create an innovative communication skills exercise using real volunteer patients and physician co-teachers for students to practice communication skills while receiving immediate feedback.

METHOD

This is a mixed methods study where second-year medical students participated in the communication skills exercise with real patients and physician co-teachers giving immediate feedback. Clinical scenarios reflected the patients' actual experiences. Students acted out roles as physicians. Physicians co-taught with the patients and gave immediate feedback to students. Students completed an anonymous written survey at the end of the exercise. Qualitative and quantitative responses were recorded. Student feedback from the 2014 surveys was used to modify the teaching designs to increase active role play opportunities by having only two students in each group and doubling the number of stations with real patients.

RESULTS

Students rated the overall exercise and the utility of patient volunteers in learning how to communicate on a Likert scale of 1-5, where in this medical school traditionally 1 is excellent and 5 is poor. In 2014, the exercises were rated with a mean score of 1.47 (SD 0.621). In 2015, the exercises were rated with a mean score of 1.03 (SD 0.62). In 2016, the exercises were rated with a mean score of 1.27 (SD 0.52). ANOVA analysis (p=0.002) and Bonferroni corrections indicate a statistically significant difference between combined mean scores of the exercise in 2014 and 2015 (p=0.001). No difference was shown between 2014 and 2016 or 2015 and 2016.

CONCLUSIONS

Medical students rated practicing communication skills with real patient volunteers and physician co-teachers giving immediate feedback in their preclinical years very highly. Student feedback indicated that they preferred active roles and increased opportunities to practice their communication skills.

Academic Performance on First-Year Medical School Exams: How Well Does It Predict Later Performance on Knowledge-Based and Clinical Assessments?

Number of appearances in the bottom quartile of 1st-year medical school exams were used to represent the extent to which students were having academic difficulties. Medical educators have long expressed a desire to have indicators of medical student performance that have strong predictive validity. Predictors traditionally used fell into 4 general categories: demographic (e.g., gender), other background factors (e.g., college major), performance/aptitude (e.g., medical college admission test scores), and noncognitive factors (e.g., curiosity). These factors, however, have an inconsistent record of predicting student performance. In comparison to traditional predictive factors, we sought to determine the extent to which academic performance in the 1st-year of medical school, as measured by examination performance in the bottom quartile of the class in 7 required courses, predicted later performance on a variety of assessments, both knowledge based (e.g., United States Medical Licensing Examination Step 1 and Step IICK) and clinical skills based (e.g., clerkship grades and objective structured clinical exam performance). Of all predictors measured, number of appearances in the bottom quartile in Year 1 was the most strongly related to performance in knowledge-based assessments, as well as clinically related outcomes, and, for each outcome, bottom-quartile performance accounted for additional variance beyond that of the traditional predictors. Low academic performance in the 1st year of medical school is a meaningful risk factor with both predictive validity and predictive utility for low performance later in medical school. The question remains as to how we can incorporate this indicator into a system of formative assessment that effectively addresses the challenges of medical students once they have been identified.

Working up rectal bleeding in adult primary care practices

RATIONALE, AIMS AND OBJECTIVES

Variation in the workup of rectal bleeding may result in guideline-discordant care and delayed diagnosis of colorectal cancer. Accordingly, we undertook this study to characterize primary care clinicians' initial rectal bleeding evaluation.

METHODS

We studied 438 patients at 10 adult primary care practices affiliated with three Boston, Massachusetts, academic medical centres and a multispecialty group practice, performing medical record reviews of subjects with visit codes for rectal bleeding, haemorrhoids or bloody stool. Nurse reviewers abstracted patients' sociodemographic characteristics, rectal bleeding-related symptoms and components of the rectal bleeding workup. Bivariate and multivariable logistic regression models examined factors associated with guideline-discordant workups.

RESULTS

Clinicians documented a family history of colorectal cancer or polyps at the index visit in 27% of cases and failed to document an abdominal or rectal examination in 21% and 29%. Failure to order imaging or a diagnostic procedure occurred in 32% of cases and was the only component of the workup associated with guideline-discordant care, which occurred in 27% of cases. Compared with patients at hospital-based teaching sites, patients at urban clinics or community health centres had 2.9 (95% confidence interval 1.3-6.3) times the odds of having had an incomplete workup. Network affiliation was also associated with guideline concordance.

CONCLUSION

Workup of rectal bleeding was inconsistent, incomplete and discordant with guidelines in one-quarter of cases. Research and improvements strategies are needed to understand and manage practice and provider variation.

Measuring the Impact of Longitudinal Faculty Development: A Study of Academic Achievement

PURPOSE

Although faculty development programs in medical education have increased over the past two decades, there is a lack of rigorous program evaluation. The aim of this study was to determine quantifiable outcomes of Harvard Medical School's (HMS's) Fellowship in Medical Education and evaluate attainment of its goals.

METHOD

In 2005 and 2009 the authors collected curricula vitae (CVs) and conducted within-subject analysis of 42 fellowship graduates and also conducted comparison analysis between 12 academic year 2005 fellows and 12 faculty who did not participate in the program. The authors identified 10 metrics of academic advancement. CV analysis for the 42 graduates started 2 years prior to fellowship enrollment and continued for 2-year intervals until June 2009 (10 years of data collection). CV analysis for the comparison group was from 2003 to 2009. The authors also analyzed association between gender and academic outcomes.

RESULTS

Fellowship graduates demonstrated significant changes in 4 of 10 academic metrics by the end of the fellowship year: academic promotion, educational leadership, education committees, and education funding. Two metrics-educational leadership and committees-showed increased outcomes two years post fellowship, with a positive trend for promotions. Fellowship graduates significantly outpaced the comparison group in 6 of 10 metrics. Women did significantly more committee work, secured more education funding, and were promoted more often than men.

CONCLUSIONS

Findings indicate that the HMS Fellowship in Medical Education meets programmatic goals and produces positive, measurable academic outcomes. Standardized evaluation metrics of longitudinal faculty development programs would aid cross-institutional comparisons.

Delayed Workup of Rectal Bleeding in Adult Primary Care: Examining Process-of-Care Failures

BACKGROUND

Although delayed colorectal cancer diagnoses figure prominently in medical malpractice claims, little is known about the quality of primary care clinicians' workup of rectal bleeding.

METHODS

In this study, 438 patients were identified using International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes for rectal bleeding, hemorrhoids, and blood in the stool at 10 Boston adult primary care practices. Following nurse chart abstraction, physician reviewers assessed the overall quality of care and key care processes. Subjects' characteristics and physician reviewers' processes-of-care assessments were tabulated, and logistic regression models were used to examine the association of process failures with overall quality and guideline concordance.

RESULTS

Although reviewers judged the overall quality of care to be good or excellent in 337 (77%) of 438 cases, 312 (71%) patients experienced at least one process-of-care failure in the workup of rectal bleeding. Clinicians failed to obtain an adequate family history in 38% of cases, complete a pertinent physical exam in 23%, and order laboratory tests in 16%. Failure to order or perform tests, or to make follow-up plans were associated with increased odds of poor or fair care. Guideline concordance bore little relationship with quality judgments. Reviewers judged that 128 delays could have been reduced or prevented.

CONCLUSION

Process-of-care failures among adult primary care patients with rectal bleeding were frequent and associated with fair or poor quality. Educating practitioners and creating systems to ensure adequate history taking, physical examination, and processes for ordering, performing, and interpreting diagnostic tests may improve performance.

A faculty development program integrating cross-cultural care into a gastrointestinal pathophysiology tutorial benefits students, tutors, and the course

A specific faculty development program for tutors to teach cross-cultural care in a preclinical gastrointestinal pathophysiology course with weekly longitudinal followup sessions was designed in 2007 and conducted in the same manner over a 6-yr period. Anonymous student evaluations of how "frequently" the course and the tutor were actively teaching cross-cultural care were performed. The statements "This tutor actively teaches culturally competent care" and "Issues of culture and ethnicity were addressed" were significantly improved over baseline 2004 data. These increases were sustained over the 6-yr period. A tutor's overall rating as a teacher was moderately correlated with his/her "frequently" actively teaching cross-cultural care (r = 0.385, P < 0. 001). Course evaluation scores were excellent and put the course into the group of preclinical courses with the top ratings. Students in the Race in Curriculum Group asked that the program be expanded to other preclinical courses. In conclusion, from 2007 to 2012, a faculty development program for teaching cross-cultural care consistently increased the discussion of cross-cultural care in the tutorial and course over each year beginning with 2007 compared with the baseline year of 2004. Our data suggest that cross-cultural care can be effectively integrated into pathophysiology tutorials and helps improve students' satisfaction and tutors' ratings. Teaching cross-cultural care in a pathophysiology tutorial did not detract from the course's overall evaluations, which remained in the top group over the 6-yr period.

Disparities in evaluation of patients with rectal bleeding 40 years and older

BACKGROUND & AIMS

Rectal bleeding is associated with colorectal cancer. We characterized the evaluation of patients aged 40 years and older with rectal bleeding and identified characteristics associated with inadequate evaluation.

METHODS

We conducted a retrospective review of records of outpatient visits that contained reports of rectal bleeding for patients aged 40 years and older (N = 480). We studied whether patient characteristics affected whether or not they received a colonoscopy examination within 90 days of presentation with rectal bleeding. Patient characteristics included demographics; family history of colon cancer and polyps; and histories of screening colonoscopies, physical examinations, referrals to specialists at the index visit, and communication of laboratory results. Data were collected from medical records, and patient income levels were estimated based on Zip codes.

RESULTS

Nearly half of the patients presenting with rectal bleeding received colonoscopies (48.1%); 81.7% received the procedure within 90 days. A history of a colonoscopy examination was more likely to be reported in white patients compared with Hispanic or Asian patients (P = .012 and P = .006, respectively), and in high-income compared with low-income patients (P = .022). A family history was more likely to be documented among patients with private insurance than those with Medicaid or Medicare (P = .004). A rectal examination was performed more often for patients who were white or Asian, male, and with high or middle incomes, compared with those who were black, Hispanic, female, or with low incomes (P = .027). White patients were more likely to have their laboratory results communicated to them than black patients (P = .001).

CONCLUSIONS

Sex, race, ethnicity, patient income, and insurance status were associated with disparities in evaluation of rectal bleeding. There is a need to standardize the evaluation of patients with rectal bleeding.

The impact of cross-cultural interactions on medical students' preparedness to care for diverse patients

PURPOSE

Medical students who graduate from schools with diverse student populations are more likely to rate themselves as prepared to care for diverse patients compared with students who graduate from more homogenous schools. This study aimed to identify the types of cross-cultural interactions associated with students' self-rated preparedness.

METHOD

In 2010, the authors developed and administered a Web-based survey that queried Harvard Medical School students about their voluntary interethnic interactions (studying, socializing), participation in diversity-related extracurricular activities, and self-rated preparedness to care for diverse patients. The authors separated students' responses regarding interethnic interactions and participation in diversity-related extracurricular activities into high and low participation, then determined the association between these responses and those to questions about students' self-rated preparedness to care for diverse patients. They used ANOVA and Z tests of proportion to analyze their data.

RESULTS

Of 724 eligible students, 460 completed the survey (64%). Seventy-five percent (324/433) believed they were prepared to care for patients from backgrounds different from their own. Students who spent >75% of their study time with students from different backgrounds or who participated in a greater number of diversity-related extracurricular activities were more likely to rate themselves as prepared to care for diverse patients overall and to perform seven other skills.

CONCLUSIONS

Voluntary cross-cultural interactions, both studying and socializing, are associated with higher self-rated preparedness to care for patients from diverse backgrounds. Medical schools should continue to support multicultural pursuits to prepare students to become physicians sensitive to the needs of diverse patients.

Developing expert-derived rating standards for the peer assessment of lectures

PURPOSE

For peer review of teaching to be credible and reliable, peer raters must be trained to identify and measure teaching behaviors accurately. Peer rater training, therefore, must be based on expert-derived rating standards of teaching performance. The authors sought to establish precise lecture rating standards for use in peer rater training at their school.

METHOD

From 2008 to 2010, a panel of experts, who had previously helped to develop an instrument for the peer assessment of lecturing, met to observe, discuss, and rate 40 lectures, using a consensus-building model to determine key behaviors and levels of proficiency for each of the instrument's 11 criteria. During this process, the panelists supplemented the original instrument with precise behavioral descriptors of lecturing. The reliability of the derived rating standards was assessed by having the panelists score six sample lectures independently.

RESULTS

Intraclass correlation coefficients of the panelists' ratings of the lectures ranged from 0.75 to 0.96. There was moderate to high positive association between 10 of the 11 instrument's criteria and the overall performance score (r = 0.752-0.886). There were no statistically significant differences among raters in terms of leniency or stringency of scores.

CONCLUSIONS

Two relational themes, content and style, were identified within the instrument's variables. Recommendations for developing expert-derived ratings standards include using an interdisciplinary group for observation, discussion, and verbal identification of behaviors; asking members to consider views that contrast with their own; and noting key teaching behaviors for use in future peer rater training.

The third year in the first person: medical students report on their principal clinical year

PURPOSE

To obtain an accurate and detailed portrait of medical students' principal clinical year using firsthand accounts of their experiences over the course of the year.

METHOD

All 45 Harvard Medical School students at four clinical sites who were training in 2005-2006 under three different pedagogical models provided open-ended responses to a monthly check-in asking them for brief descriptions of any interesting or memorable experiences associated with their clerkships. Associations with gender, time of year, rotation, and clerkship model were also studied.

RESULTS

A total of 770 incidents were collected, and these were coded for positivity-negativity and content. Five broad theme areas were identified: stories about physicians (e.g., physicians giving instruction, acting as role models), medical students and their behavior (feelings of uncertainty or being useful, of workload), patients and patient care (e.g., learning by doing or observing, forming bonds with patients, memorable patients, treating patients over time), groups and group climate (e.g., effectiveness of teams, informal groups, comparison of services), and content themes (e.g., birth, death, cancer, bad news). Two-thirds of all stories were coded as positive.

CONCLUSIONS

These third-year medical students often framed their experiences positively, finding learning lessons even in stressful or unpleasant events. Their stories also reflect relatively consistent orientations toward patients and patient care (e.g., biomedical versus patient-centered). The authors believe these incidents reflect the emerging professional identities of medical students; educators can use these to help students reflect on the kind of physician they aspire to become.

Using medical students to enhance curricular integration of cross-cultural content

We hypothesized that an interested medical student group would be helpful in reviewing tutorial cases and giving relevant feedback on the curricular integration of cross-cultural content using case triggers in a preclinical gastrointestinal pathophysiology course. Self-selected student leaders (n = 9) reviewed pre-existing problem-based learning tutorial cases (n = 3) with cross-cultural triggers, and provided narrative feedback to course faculty. The cases were modified and used for the entire class in the following 2 years. Participating course students' comments and teaching faculty feedback were also noted. Outcomes were a change in case content, student global evaluations of the course, and self-reported faculty comfort with teaching the cases. All three tutorial cases were reviewed by a separate group of 2-3 students. Major and minor revisions were made to each case based on the student feedback. These cases were used in 2007 and 2008 and were the major change to the course during that time. Overall course evaluation scores improved significantly from 2006 to 2008 (p = 0.000). Tutors (n = 22 in 2007; n = 23 in 2008) expressed relief during tutor meetings that students had reviewed the cases. A general framework for eliciting student feedback on problem-based cases was developed. Student feedback, consisting of self-selected students' case reviews and solicited course and tutor comments, added value to a curricular reform to improve the integration of cross-cultural content into a problem-based learning curriculum. Our study underscores the fundamental link between teachers and students as partners in curricular development.

Psychiatric OSCE performance of students with and without a previous core psychiatry clerkship

OBJECTIVE

The OSCE has been demonstrated to be a reliable and valid method by which to assess students' clinical skills. An OSCE station was used to determine whether or not students who had completed a core psychiatry clerkship demonstrated skills that were superior to those who had not taken the clerkship and which areas discriminated between clerkship completers and noncompleters.

METHODS

One hundred thirty-six students took a 48-item, fourth-year OSCE with one psychiatry station. Sixty-three (46%) had already completed psychiatry, and 70 (51%) had not, with three unknown. Students were to take histories, perform mental status examinations, assess dangerousness, and propose the differential diagnosis and treatment plans.

RESULTS

Nine items differed significantly between completers and noncompleters, six concerning phenomenology and mental status and three concerning differential diagnosis. There were no differences regarding history, communication skills, or recommended interventions.

CONCLUSION

Students may learn history taking, communication, and treatment planning in many settings. However, for the mental status examination, phenomenology, and differential diagnosis, completing a psychiatry clerkship was associated with better OSCE performance.

Integration of racial, cultural, ethnic, and socioeconomic factors into a gastrointestinal pathophysiology course

BACKGROUND & AIMS

Our study describes a faculty development program to encourage the integration of racial, cultural, ethnic, and socioeconomic factors such as obesity, inability to pay for essential medications, the use of alternative medicine, dietary preferences, and alcoholism in a gastrointestinal pathophysiology course.

METHODS

We designed a 1-hour faculty development session with longitudinal reinforcement of concepts. The session focused on showing the relevance of racial, ethnic, cultural, and socioeconomic factors to gastrointestinal diseases, and encouraged tutors to take an active and pivotal role in discussion of these factors. The study outcome was student responses to course evaluation questions concerning the teaching of cultural and ethnic issues in the course as a whole and by individual tutorials in 2004 (pre-faculty development) and in 2006 to 2008 (post-faculty development).

RESULTS

Between 2004 and 2008, the proportion of students reporting that "Issues of culture and ethnicity as they affect topics in this course were addressed" increased significantly (P = .000). From 2006 to 2008, compared with 2004, there was a significant increase in the number of tutors who "frequently" taught culturally competent care according to 60% or greater of their tutorial students (P = .003). The tutor's age, gender, prior tutor experience, rank, and specialty did not significantly impact results.

CONCLUSIONS

An innovative faculty development session that encourages tutors to discuss racial, cultural, ethnic, and socioeconomic issues relevant to both care of the whole patient and to the pathophysiology of illness is both effective and applicable to other preclinical and clinical courses.

A faculty development program to train tutors to be discussion leaders rather than facilitators

PURPOSE

During 2003, 2004, and 2005, the role of 70 tutors was changed from that of facilitator to discussion leader, in a preclinical PBL learning course, Gastrointestinal Pathophysiology, by use of three key business school teaching strategies: questions, summaries, and schematics. The purpose of this study was to learn what difference this new approach made.

METHOD

During each of the three study years, 171 (2003), 167 (2004), and 170 (2005) students were given Likert-scale attitudinal questionnaires to rate whether their tutors encouraged student direction of the tutorials and whether the summaries and closure schematics benefited their learning. Students' overall course evaluations and mean USMLE scores were quantitatively analyzed, pre- and postintervention. A variety of statistical tests were used to assess the statistical significance of means at the confidence level of .05.

RESULTS

In the third year of the program, student ratings indicated that their tutors were significantly better at encouraging student direction of the tutorials than in the first year (P < .05). The students reported that the tutorial made a more important contribution to their learning (P < .05), and the course objectives were better stated (P = .038) and better met (P = .007). Overall satisfaction with the course also improved significantly (P = .006). Part I gastrointestinal system mean scores of the USMLE showed a statistically significant increase in 2005 compared with 2001 or 2002.

CONCLUSIONS

The tutor as a discussion leader who questions, summarizes, and uses schematics to illustrate concepts had a significant and positive impact on learning in tutorials, achieving course objectives, improving overall course satisfaction, and increasing a standardized national exam's mean score.