Effect of clinical teaching on student performance during a medicine clerkship

The Use and Teaching of Telemedicine in Allergy/Immunology Training Programs

PURPOSE OF REVIEW

The use of telemedicine has greatly increased since the onset of the coronavirus disease 2019 (COVID-19) pandemic. This review discusses the types of telemedicine, current telehealth curricula in medical education, and benefits and disadvantages of incorporation of telemedicine into Allergy/Immunology training programs.

RECENT FINDINGS

The majority of Allergists/Immunologists use telemedicine in their clinical practice with leaders in graduate medical education recommending inclusion of telemedicine in training. Fellows-in-training reported that use of telemedicine in Allergy/Immunology training during the pandemic mitigated some concerns for lack of adequate clinical experience. Still, no standardized curriculum for telemedicine training in Allergy/Immunology exists, although curricula from internal medicine and primary care residencies can provide a framework for incorporation of telemedicine training into fellowship. Benefits of telemedicine in Allergy/Immunology training include enhanced immunology training, home environment monitoring, and flexibility to reduce physician burnout while disadvantages include limited physical examination skill building and lack of a standardized curriculum. As telemedicine has been widely accepted in medicine with high patient satisfaction, it is necessary to incorporate a standardized telehealth curriculum in Allergy/Immunology fellowship training, both as a tool for patient care as well as trainee education.

Using the Modified Delphi Method to Develop a Tool to Assess Pediatric Residents Supervising on Inpatient Rounds

BACKGROUND

Consensus about behaviors that define effective supervision by residents of more junior trainees on pediatric inpatient rounds is lacking.

OBJECTIVE

Use modified Delphi method to develop a checklist of essential supervisory behaviors pediatric residents demonstrate while leading inpatient, non-ICU, nonspecialty teaching rounds and pilot the checklist.

DESIGN/METHODS

One hundred and forty-one initial candidate behaviors were identified through literature review and narrowed by local stakeholders. Forty-one behaviors were submitted to national experts identified through purposive sampling to be refined using the modified Delphi method. Participants indicated agreement with behaviors' inclusion in the checklist and whether any were nonobservable. Measures of central tendency and dispersion were used to identify consensus with a behavior's inclusion in the next cycle. A criterion was eliminated if >25% of experts felt it was not observable. Cycles continued until consensus was reached on ≤20 behaviors. The resulting checklist was piloted at 2 hospitals.

RESULTS

After 2 modified Delphi cycles, consensus was obtained on 18 behaviors for inclusion in the final checklist. These were spread across 3 subcategories: teaching, leadership, and patient safety. In the pilot, the checklist of behaviors discriminated between residents with differing levels of competency in supervising trainees on inpatient rounds. For residents who had the checklist completed by 2 faculties, there was 75% agreement in behaviors observed.

CONCLUSION(S)

Based on literature review, local stakeholder input, and consensus of national experts through the modified Delphi method, we created and piloted a checklist of observable behaviors characteristic of effective clinical supervision by pediatric residents leading trainee teams on inpatient, non-ICU, nonspecialty rounds.

Teaching when Time Is Limited: a Resident and Fellow as Educator Video Module

BACKGROUND

Research has demonstrated that residents and fellows (hereafter referred to as "residents") play a significant role in the education of medical students. However, residents often feel unprepared to teach effectively and efficiently in busy clinical environments.

ACTIVITY

The Residents as Educators Committee at the Medical College of Wisconsin sought to create an online module for residents to promote five key elements of teaching medical students in a busy clinical environment when time is limited. The module includes a narrated presentation highlighting key attributes of excellent clinical teachers as well as video clips presenting teaching pearls from award-winning resident educators. All incoming residents during the 2015-2016 academic year were required to view the module and complete a mandatory post-test, representing over 80 specialties and subspecialties.

RESULTS

A total of 325 residents viewed the module and took the required post-test. Of this group, 294 residents (91.4%) completed the optional evaluation. The module was rated highly in terms of content, format, and likelihood to change teaching practices.

DISCUSSION

The teaching provided by residents is vital to the success of future generations of medical students. The online module developed at the Medical College of Wisconsin may be useful to a larger audience of residents at other institutions. Additionally, further studies could determine the effectiveness of the module by examining the teaching evaluations of residents before and after viewing the module.

The role of study strategy in motivation and academic performance of ethnic minority and majority students: a structural equation model

Underperformance among ethnic minority students has been reported in several studies. Autonomous motivation (acting out of true interest or personal endorsement) is associated with better learning and academic performance. This study examined whether study strategy (surface, achieving, and deep) was a mediator between the type of motivation (autonomous and controlled motivation) and academic performance (GPA and clerkship performance), and whether these relations are different for students from different ethnic groups to gain a better understanding about the needed intervention/support in the curriculum. Data was gathered from 947 students at VUmc School of Medical Sciences, Amsterdam. Structural Equation Modelling was performed to test the hypothesized model: a higher autonomous motivation has a positive association with academic performance through deep and achieving strategy, and has a negative association with performance through surface strategy. The model with the outcome variables GPA and clerkship performance had a good fit (n = 618; df = 1, RMSEA = 0.000, p = 0.43). The model for the ethnic majority and minority groups was significantly different (p < 0.025). In this study, autonomous motivation had a positive association with GPA through achieving strategy for the ethnic majority students only. It might be that the size of the minority groups was too small to detect differences or that other factors mediate these relations in ethnic minority students. Qualitative research is needed to identify other factors influencing the academic performance of ethnic minority students and what they experience during their education, in order to support their learning in the right manner.

The self-regulated learning of medical students in the clinical environment - a scoping review

BACKGROUND

Self-regulated learning is the individual's ability to effectively use various strategies to reach their learning goals. We conducted this scoping review to explore what has been found regarding self-regulated learning in the clinical environment and how this was measured.

METHODS

Using Arksey and O'Malley's five-stage framework, we searched three medical and educational databases as well as Google Scholar for literature on the self-regulated learning of medical students in the clinical environment published between 1966 and February 2017. After results were screened and relevant studies were identified, the data was summarised and discursively reported.

RESULTS

The search resulted in 911 articles, with 14 articles included in the scoping review after the inclusion criteria was applied. Self-regulated learning was explored in these studies in various ways including qualitative, quantitative and mixed methods. Three major findings were found: 1) levels of self-regulated learning change in the clinical environment, 2) self-regulated learning is associated with academic achievement, success in clinical skills and mental health and 3) various factors can support self-regulated learning levels in medical students.

CONCLUSIONS

Most of articles exploring the self-regulated learning of medical students during the clinical years have been published in the last 5 years, suggesting a growing interest in the area. Future research could explore the self-regulated learning levels of medical students during the clinical years using a longitudinal approach or through the use of novel qualitative approaches.

We need to talk: clinical competency committees in the key of c(onversation)

None

Influence of motivation, self-efficacy and situational factors on the teaching quality of clinical educators

BACKGROUND

Being exposed to good teachers has been shown to enhance students' knowledge and their clinical performance, but little is known about the underlying psychological mechanisms that provide the basis for being an excellent medical teacher. Self-Determination Theory (SDT) postulates that more self-regulated types of motivation are associated with higher performance. Social Cognitive Theory (SCT) focuses on self-efficacy that has been shown to be positively associated with performance. To investigate the influences of different types of teaching motivation, teaching self-efficacy, and teachers' perceptions of students' skills, competencies and motivation on teaching quality.

METHODS

Before the winter semester 2014, physicians involved in bedside teaching in internal medicine at the University Medical Center Hamburg-Eppendorf completed a questionnaire with sociodemographic items and instruments measuring different dimensions of teaching motivation as well as teaching self-efficacy. During the semester, physicians rated their perceptions of the participating students who rated the teaching quality after each lesson. We performed a random intercept mixed-effects linear regression with students' ratings of teaching quality as the dependent variable and students' general interest in a subject as covariate. We explored potential associations between teachers' dispositions and their perceptions of students' competencies in a mixed-effects random intercept logistic regression.

RESULTS

94 lessons given by 55 teachers with 500 student ratings were analyzed. Neither teaching motivation nor teaching self-efficacy were directly associated with students' rating of teaching quality. Teachers' perceptions of students' competencies and students' general interest in the lesson's subject were positively associated with students' rating of teaching quality. Physicians' perceptions of their students' competencies were significantly positively predicted by their teaching self-efficacy.

CONCLUSIONS

Teaching quality might profit from teachers who are self-efficacious and able to detect their students' competencies. Students' general interest in a lesson's subject needs to be taken into account when they are asked to evaluate teaching quality.

The Use of Rubrics in the Clinical Evaluation of Podiatric Medical Students: Objectification of the Subjective Experience

BACKGROUND

We assessed the differences in podiatric medical students' clinical professionalism objective scores (CPOSs) by comparing a previous nonrubric evaluation tool with a more recently implemented objective-centered rubric evaluation tool. This type of study has never been performed or reported on in the podiatric medical education literature.

METHODS

We conducted a retrospective analysis of 89 third-year podiatric medical students between academic years 2010-2011 and 2011-2012. A Pearson correlation coefficient analysis was performed to compare CPOSs from the students' first (CPOS1) and second (CPOS2) rotations. A correlation analysis was performed comparing students' grade point averages (GPAs) with each of the individual CPOSs to verify the validity of the rubric evaluation tool.

RESULTS

The Pearson correlation coefficients for the relationship between 2012 CPOS1 and CPOS2 and GPA were r = 0.233 (P ≤ .093) and r = 0.290 (P < .035) and for the relationship between 2013 CPOS1 and CPOS2 and GPA were r = 0.525 (P = .001) and r = 0.730 (P < .001).

CONCLUSIONS

These findings suggest that the use of a rubric in the evaluation of podiatric medical students' CPOSs is correlated with their GPAs, and CPOS2 demonstrated a higher correlation than CPOS1. We believe that implementation of the rubric evaluation tool has increased the accuracy of the evaluation of podiatric medical students with respect to CPOSs.

Is Reflective Ability Associated With Professionalism Lapses During Medical School?

PURPOSE

Recently, many have argued that learning to reflect on one's experiences is a critical component of professional identity formation and of professionalism. However, little empirical evidence exists to support this claim. This study explored the association between reflective ability and professionalism lapses among medical students.

METHOD

The authors conducted a retrospective case-control study of all students who matriculated at Indiana University School of Medicine from 2001 to 2009. The case group (n = 70) included those students who had been cited for a professionalism lapse during medical school; the students in the control group (n = 230) were randomly selected from the students who had not been cited for a professionalism lapse. Students' professionalism journal entries were scored using a validated rubric to assess reflective ability. Mean reflection scores were compared across groups using t tests, and logistic regression analysis was used to assess the relationship between reflective ability and professionalism lapses.

RESULTS

Reflection scores for students in the case group (2.46 ± 1.05) were significantly lower than those for students in the control group (2.82 ± 0.83) (P = .01). A lower reflection score was associated with an increased likelihood that the student had been cited for a professionalism lapse (odds ratio = 1.56; P < .01).

CONCLUSIONS

This study revealed a significant relationship between reflective ability and professionalism, although further study is needed to draw any conclusions regarding causation. These findings provide quantitative evidence to support current anecdotal claims about the relationship between reflection and professionalism.

Pre-admission criteria and pre-clinical achievement: Can they predict medical students performance in the clinical phase?

RATIONAL

Various factors affect medical students' performance during clinical phase. Identifying these factors would help in mentoring weak students and help in selection process for residency programmes.

OBJECTIVE

Our study objective is to evaluate the impact of pre-admission criteria, and pre-clinical grade point average (GPA) on undergraduate medical students' performance during clinical phase.

METHOD

This study has a cross-sectional design that includes fifth- and sixth-year female medical students (71). Data of clinical and pre-clinical GPA in medical school and pre-admission to medical school tests scores were collected.

RESULTS

A significant correlation between clinical GPA with the pre-clinical GPA was observed (p < 0.05). Such significant correlation was not seen with other variables under study. A regression analysis was performed, and the only significant predictor of students clinical performance was the pre-clinical GPA (p < 0.001). However, no significant difference between students' clinical and pre-clinical GPA for both cohorts was observed (p > 0.05).

CONCLUSION

Pre-clinical GPA is strongly correlated with and can predict medical students' performance during clinical years. Our study highlighted the importance of evaluating the academic performances of students in pre-clinical years before they move into clinical years in order to identify weak students to mentor them and monitor their progress.

Validation of the 25-Item Stanford Faculty Development Program Tool on Clinical Teaching Effectiveness

CONSTRUCT: The 25-item Stanford Faculty Development Program Tool on Clinical Teaching Effectiveness assesses clinical teaching effectiveness.

BACKGROUND

Valid and reliable rating of teaching effectiveness is helpful for providing faculty with feedback. The 25-item Stanford Faculty Development Program Tool on Clinical Teaching Effectiveness was intended to evaluate seven dimensions of clinical teaching. Confirmation of the structure of this tool has not been previously performed.

APPROACH

This study sought to validate this tool using a confirmatory factor analysis, testing a 7-factor model and compared its goodness of fit with a modified model. Acceptability of the use of the tool was assessed using a 6-item survey, completed by final year medical students (N = 119 of 156 students; 76%).

RESULTS

The testing of the goodness of fit indicated that the 7-factor model performed poorly, χ(2)(254) = 457.4, p < .001 (root mean square error of approximation [RMSEA] = 0.08, comparative fit index [CFI] = 0.91, non-normed fit index [NNFI] = 0.89). Only standardized root mean square residual (SRMR) indicated acceptable fit (0.06). Further exploratory analysis identified 10 items that cross-loaded on 2 factors. The remainder of the items loaded on factors as originally intended. By removing these 10 items, repeat confirmatory factor analysis on the modified 15-item, 5-factor model demonstrated a better fit than the original model: SRMR = 0.075, NNFI = 0.91, χ(2)(80) = 150.1, p < .001; RMSEA = 0.09; CFI = 0.93. Although 75% of the participants stated they were willing to fill the tool on their preceptors on a biweekly basis, only 25% were willing to do so on a weekly basis.

CONCLUSIONS

Our study failed to confirm factor structure of the 25-item tool. A modified tool with fewer, more conceptually distinct items was best fit by a 5-factor model. Further, the acceptability of use for the 25-item tool may be poor for rotations with a new preceptor weekly. The abbreviated tool may be preferable in that setting.

Meeting international standards: a cultural approach in implementing the mini-CEX effectively in Indonesian clerkships

Abstract Background: Medical schools all over the world try to adapt their programs to meet international standards. However, local culture might hamper innovation attempts.

AIMS

To describe challenges in implementing the mini-CEX in Indonesia and investigate its effect on students' clinical competence.

METHODS

The study was conducted in the Internal Medicine and Neurology departments of the Universitas Gadjah Mada, Indonesia. Implementing the mini-CEX into the existing curriculum, while taking the Indonesian culture into account, implied a shift from group to individual feedback. We compared students' final clinical competence before (Internal Medicine n = 122, Neurology n = 183) and after (n = 183 and 186, respectively) the implementation of the mini-CEX, using a modified Objective Structured Long Examination Record (OSLER). The Mann-Whitney test was used to analyze the data.

RESULTS

We took power distance and individualism into account to facilitate the implementation process. After implementing the mini-CEX, the OSLER results were significant higher in Internal Medicine (p < 0.05). However, no differences were found in Neurology.

CONCLUSION

By managing the innovation process carefully and taking culture and local context into account, the mini-CEX can be implemented without changing the underlying concept. The shift from group to individual feedback seems to have a positive effect on student learning.

Faculty development for hospitalists: structured peer observation of teaching

BACKGROUND

Hospitalists provide much of the clinical teaching in internal medicine, yet formative feedback to improve their teaching is rare.

METHODS

We developed a peer observation, assessment, and feedback program to improve attending hospitalist teaching. Participants were trained to identify 10 optimal teaching behaviors using a structured observation tool that was developed from the validated Stanford Faculty Development Program clinical teaching framework. Participants joined year-long feedback dyads and engaged in peer observation and feedback on teaching. Pre- and post-program surveys assessed confidence in teaching, performance of teaching behaviors, confidence in giving and receiving feedback, attitudes toward peer observation, and overall satisfaction with the program.

RESULTS

Twenty-two attending hospitalists participated, averaging 2.2 years (± 2.1 years standard deviation [SD]) experience; 15 (68%) completed pre- and post-program surveys. Confidence in giving feedback, receiving feedback, and teaching efficacy increased (1 = strongly disagree, 5 = strongly agree, mean ± SD): "I can accurately assess my colleagues' teaching skills," (pre = 3.2 ± 0.9 vs post = 4.1 ± 0.6, P < 0.01), "I can give accurate feedback to my colleagues" (pre = 3.4 ± 0.6 vs post = 4.2 ± 0.6, P < 0.01), and "I am confident in my ability to teach students and residents" (pre = 3.2 ± 0.9 vs post = 3.7 ± 0.8, P = 0.026).

CONCLUSIONS

Peer observation and feedback of teaching increases hospitalist confidence in several domains that are essential for optimizing teaching. Further studies are needed to examine if educational outcomes are improved by this program.

Performance of residents serving as clinical teachers: a student-based assessment

BACKGROUND

Little is known about the competences of residents as clinical teachers in African health care institutions.

OBJECTIVE

We evaluated the clinical teaching skills of internal medicine residents from the perspective of medical students in a tertiary teaching institution in Africa.

METHODS

We used the augmented Stanford Faculty Development Program Questionnaire, which has evidence of validity and reliability. To avoid a Hawthorne effect, students completed the questionnaire anonymously and confidentially after clinical teaching sessions by residents. A minimum score of 4 on a scale of 1 to 5 was defined a priori as possession of good clinical teaching skills.

RESULTS

Sixty-four medical students assessed all 20 internal medicine residents in the Department of Medicine, University of Ibadan. Mean performance scores for the domains ranged from 3.07 to 3.66. Residents performed best in creating a good learning climate and worst in the promotion of understanding and retention. Sex of the resident, duration of residency, and rank had no significant impact (.09 < P < .94) on their teaching skills.

CONCLUSIONS

Consistent with other observations in the literature, residents' clinical teaching skills were suboptimal, particularly in their ability to promote understanding and retention. To enhance these skills, we recommend the integration of appropriately tailored programs to teach pedagogic skills programs in residency training.

Can incorporating inpatient overnight work hours into a pediatric clerkship improve the clerkship experience for students?

PURPOSE

After incorporating medical students into pediatric resident night-float teams, the authors studied the effects of the new schedule on (1) cognitive performance, (2) number of new admissions, (3) clerkship satisfaction, and (4) amount and quality of resident teaching.

METHOD

Part 1 was a retrospective historical controls study. The intervention was a schedule change that eliminated inpatient call. The historical control group had a four-week inpatient schedule of daytime hours plus five calls (DT+C). The comparison group had a schedule of three weeks of daytime hours plus five consecutive overnight shifts (DT+OS). National Board of Medical Examiners (NBME) Pediatrics Subject Exam scores, number of admission history and physicals (HPEs), and clerkship satisfaction data from both groups were compared. Part 2 was a two-item survey with open-ended comments that measured perceptions of resident teaching time and quality of resident teaching (QRT) from students on the DT+OS schedule.

RESULTS

DT+OS students had a significantly increased number of HPEs (t=2.17; P=.03) compared with the DT+C group (mean=7.49, standard deviation [SD]=3.34 in DT+OS versus mean=6.11, SD=2.95 in DT+C). The paired samples t test showed that students rated QRT significantly higher when on overnights than when they were on daytime hours (t=2.47; P=.02). There were no differences in satisfaction or NBME scores.

CONCLUSION

Overnight work hours for medical students increased clerkship capacity while maintaining student satisfaction and cognitive performance. Added benefits included increased clinical experience and improved QRT.

Impact of increased authenticity in instructional format on preclerkship students' performance: a two-year, prospective, randomized study

PURPOSE

To address whether increasingly authentic instructional formats are more effective in improving preclerkship medical students' performance.

METHOD

From 2007 to 2009, the authors conducted a prospective, randomized, crossover study with second-year medical students in a clinical reasoning course at the Uniformed Services University of the Health Sciences. The authors randomly assigned students to one of three cohorts and used instructional formats of differing authenticity (paper, DVD, standardized patient) to teach three subject areas (abdominal pain, anemia, polyuria). Each cohort received one instructional format for each subject area. The authors collected outcome measures (objective structured clinical exam, video quiz, and essay exam scores) at the end of each academic year. They stratified the students into tertiles by first-year grade point average to investigate the impact of instructional formats on learners of different abilities.

RESULTS

Outcomes for students in the top tertile improved with increased authenticity of the instructional format compared with outcomes for students in the middle and bottom tertiles (0.188 versus -0.038 and -0.201, P=.001 and .027, respectively). However, outcomes for students in the bottom tertile decreased when students were given only the paper case, compared with the middle and top tertiles (-0.374 versus 0.043 and 0.023, respectively, P=.001), but subsequently improved with more authentic instructional formats.

CONCLUSIONS

The authors could not demonstrate that increased authenticity of the instructional format resulted in improved learner performance. However, they believe that there may be some benefit to tailoring preclerkship clinical education based on students' ability.

Impact of a Resident-as-Teacher Workshop on Teaching Behavior of Interns and Learning Outcomes of Medical Students

BACKGROUND

Residents and interns are recognized as important clinical teachers and mentors. Resident-as-teacher training programs are known to improve resident attitudes and perceptions toward teaching, as well as their theoretical knowledge, skills, and teaching behavior. The effect of resident-as-teacher programs on learning outcomes of medical students, however, remains unknown. An intervention cohort study was conducted to prospectively investigate the effects of a teacher-training workshop on teaching behavior of participating interns and on the clerkship learning outcomes of instructed fourth-year medical students.

METHODS

The House Officer-as-Teacher Training Workshop was implemented in November 2009 over 1.5 days and attended by all 34 interns from one teaching hospital. Subsequently, between February and August 2010, 124 fourth-year medical students rated the observable teaching behavior of interns during 6-week general surgery clerkships at this intervention hospital as well as at 2 comparable hospitals serving as control sites. Ratings were collected using an anonymous 15-item Intern Clinical Teaching Effectiveness Instrument. Student achievement of clerkship learning outcomes during this period was evaluated using a validated and centralized objective structured clinical examination.

RESULTS

Medical students completed 101 intern clinical teaching effectiveness instruments. Intern teaching behavior at the intervention hospital was found to be significantly more positive, compared with observed behavior at the control hospitals. Objective structured clinical examination results, however, did not demonstrate any significant intersite differences in student achievement of general surgery clerkship learning outcomes.

CONCLUSIONS

The House Officer-as-Teacher Training Workshop noticeably improved teaching behavior of surgical interns during general surgery clerkships. This improvement did not, however, translate into improved achievement of clerkship learning outcomes by medical students during the study period.

Does the authenticity of preclinical teaching format affect subsequent clinical clerkship outcomes? A prospective randomized crossover trial

BACKGROUND

Educational theories predict conflicting results for the effect of increasing the authenticity of the teaching format of complex information on educational outcomes. We sought to determine the effect of increasingly authentic small-group, preclerkship teaching format on clerkship outcomes to further enlighten this debate.

SUMMARY

Students enrolled in a prospective randomized crossover trial that involved three content areas. For each content area, three teaching formats were tested. Participants were randomized to teaching format by content area. Clerkship outcomes were performance on an objective structured clinical exam, a DVD exam, internal medicine clerkship grades, and performance on the subject examination. The data were analyzed using a multivariate analysis of covariance. One hundred and thirty-three (78%) students participated. Teaching format did not have a statistically significant effect on any of the specified clerkship outcomes. However, number of patients seen was significantly associated with higher scores in respective outcomes by topic.

CONCLUSIONS

Second-year teaching format did not directly influence subsequent clerkship performance. Our study adds to the literature by demonstrating that the authenticity of preclinical teaching format does not appear to matter for clerkship performance; however, the number of actual patients seen does appear to influence related clerkship outcomes.

Measuring the effectiveness of pharmacology teaching in undergraduate medical students

OBJECTIVE

Information overload and recent curricular changes are viewed as important contributory factors to insufficient pharmacological education of medical students. This study was designed to assess the effectiveness of pharmacology teaching in our medical school.

METHODS

The study subjects were 455 second-year medical students, class of 2010, and 26 pharmacology teachers at the National University of Mexico Medical School. To assess pharmacological knowledge, students were required to take 3 multiple-choice exams (70 questions each) as part of their evaluation in the pharmacology course. A 30-item questionnaire was used to explore the students' opinion on teaching. Pharmacology professors evaluated themselves using a similar questionnaire. Students and teachers rated each statement on a 5-point Likert scale.

RESULTS

The groups' exam scores ranged from 54.5% to 90.0% of correct responses, with a mean score of 77.3%. Only 73 (16%) of 455 students obtained an exam score of 90% and higher. Students' evaluations of faculty and professor self-ratings were very high (90% and 96.2%, of the maximal response, respectively). Student and professor ratings were not correlated with exam scores (r = 0.291).

CONCLUSIONS

Our study shows that knowledge on pharmacology is incomplete in a large proportion of second-year medical students and indicates that there is an urgent need to review undergraduate training in pharmacology. The lack of relationship between the subjective ratings of teacher effectiveness and objective exam scores suggests the use of more demanding measures to assess the effectiveness of teaching.

Academic advancement of clinician educators: why is it so difficult?

The role of clinician educators (CEs) in institutions and medical centres continues to be vital without any doubt. Although there has been more than a century since Sir William Osler established the role of the CE and the tradition by encouraging bed-side teaching, there is still a lack of consensus on the attributes that define a 'clinician-educator'. The concept of a superior clinician who is also a dedicated teacher seems to fit the description of a CE but most often seems insufficient to support the CE's academic advancement.

Effect of medical students' experiences with residents as teachers on clerkship assessment

BACKGROUND

We assessed the impact of resident teaching in medical students' overall assessment of an obstetrics and gynecology clerkship.

METHODS

Between September 2007 and March 2010, third-year medical students completed a questionnaire at the end of their clerkship. Questions covered the number of topics they recalled learning about from residents, perceptions of the quality of education received from residents, perceptions of the experience with the residents as teachers, and overall rating of the obstetrics and gynecology clerkship.

RESULTS

Questionnaires were completed by all medical students (N  =  74), with 63 students returning complete surveys. There was a small correlation between the number of topics students recalled learning about from residents and ratings of quality of education (r  =  0.249, P  =  .03), a small correlation between the number of topics students recalled learning about from residents and level of satisfaction with the clerkship (r  =  0.262, P  =  .04), and a larger correlation between assessment of the overall experience with residents as teachers and ratings of quality of education received from residents (r  =  0.687, P < .001). Overall experience with residents as teachers and perceived quality of education from residents were both associated with satisfaction with the clerkship (r  =  0.756, P < .001 and r  =  0.603, P < .001, respectively). A regression model with these variables accounted for 58% of the variability in students' ratings of the clerkship (F  =  27.103, P < .001, R  =  .761, R(2)  =  .579). The only significant independent variable was assessment of the overall experience with residents as teachers (t  =  5.350, P < .001).

CONCLUSION

Quality of interactions between residents and medical students is a key factor in medical students' assessment of their clerkship.

Assessing the quality of clinical teaching: a preliminary study

OBJECTIVES

Evaluations in the clinical arena are fraught with problems. Current assessments of clinical teaching typically measure attributes of clinical teachers in overly broad terms, are often subjective and often succumb to the halo effect. This is in contradistinction to measurements of lectures, workshops or online educational content, which can more readily be assessed using objective criteria. As a result, clinical evaluations are often insufficient to provide focused feedback, guide faculty development or identify specific areas for clinical teachers to implement change and improvement. The aim of our study was to offset these limitations.

METHODS

We developed a structured, 15-item objective structured clinical examination (OSCE)-type checklist of discrete teaching behaviours intended to be: (i) observable; (ii) applicable to multiple disciplines, and (iii) reliably identifiable. Our goal was to test and utilise this checklist as an objective assessment of clinical teaching across a range of in-patient teaching rounds experiences. During 2007-2008, pairs of external raters on two separate occasions observed nine attending physicians during actual in-patient paediatrics and internal medicine ward rounds at a large, academic medical centre. Observers documented the extent to which specific teaching behaviours did or did not occur.

RESULTS

The internal consistency of the 15-item checklist was good (alpha = 0.85). A two-facet, partially nested G study found the generalisability of ratings to be generally acceptable, but inter-rater reliability varied greatly between occasions and across individual checklist items.

CONCLUSIONS

Despite attempts to identify discrete and observable target behaviours, placing observers on rounds to detect these behaviours may not be as straightforward as it would seem. Clinical teaching may be a more inherently subjective process, based on different teaching styles of faculty staff. However, a set of objective checklist items to be completed by trained observers on teaching rounds holds promise as a potentially viable means of identifying strengths and weaknesses of clinical instruction. Further research is needed to define what constitutes quality clinical teaching, as well as the most reliable method for assessing it.

Service versus education: finding the right balance: a consensus statement from the Council Of Emergency Medicine Residency Directors 2009 Academic Assembly "Question 19" working group

Many emergency medicine (EM) residency programs have recently received citations for their residents' responses to Question 19 of the Accreditation Council on Graduate Medical Education annual survey, which asks residents to rate their program's emphasis on clinical education over service obligations. To the best of our knowledge, no prior investigations or consensus statements exist that specifically address the appropriate balance between educational activity and clinical service in EM residency training. The objective of this project was to create a consensus statement based on the shared insights of academic faculty and educators in EM, with specific recommendations to improve the integration of education with clinical service in EM residency training programs. More than 80 EM program directors (PDs), associate and assistant PDs, and other academic EM faculty attending an annual conference of EM educators met to address this issue in a discussion session and working group. Participants examined the current literature on resident service and education and shared with the conference at large their collective insight and experience and possible solutions to this challenge. A consensus statement of specific recommendations and effective educational techniques aimed at balancing service and education requirements was created, based on the contributions of a diverse group of academic emergency physicians. Recommendations included identifying the teachable moment in all clinical service; promoting resident understanding of program goals and expectations from the beginning; educating residents about the ACGME resident survey; and engaging hospitals, institutional graduate medical education departments, and residents in finding solutions.

Internal medicine clerkship characteristics associated with enhanced student examination performance

PURPOSE

To determine which internal medicine (IM) clerkship characteristics are associated with better student examination performance.

METHOD

The authors collected data from 17 U.S. medical schools (1,817 students) regarding characteristics of their IM clerkships, including structural characteristics, pedagogical approaches, patient contact, and clinical teacher characteristics. Outcomes of interest were postclerkship National Board of Medical Examiners (NBME) subject examination score, United States Medical Licensing Examination (USMLE) 2 score, and change in score from USMLE 1 to 2. To examine how associations of various clerkship characteristics and examination performance may differ for students of different prior achievement, the authors categorized students into those who scored in the top (1/4) of the cohort on USMLE 1 and the bottom (1/4). The authors conducted analyses at both the school and the individual student levels.

RESULTS

In school-level analyses (using a reduced four-variable model), independent variables associated with higher NBME subject examination score were more small-group hours/week and use of community-based preceptors. Greater score increase from USMLE 1 to 2 was associated with students caring for more patients/day. Several variables were associated with enhanced student examination performance at the student level. The most consistent finding was that more patients cared for per day was associated with higher examination performance. More structured learning activities were associated with higher examination scores for students with lower baseline USMLE 1 achievement.

CONCLUSION

Certain clerkship characteristics are associated with better student examination performance, the most salient being caring for more patients per day.

Instructiveness of feedback during clerkships: influence of supervisor, observation and student initiative

BACKGROUND

Several authors assume that the supervisor's role, observation of behaviour and students' active participation are important factors in the instructiveness of feedback.

AIM

This study aims to provide empirical evidence for these expectations.

METHODS

For two weeks, 142 clerks from eight hospitals recorded for each individual feedback event: who provided the feedback, whether the feedback was based on observation of behaviour, who initiated the feedback moment and the perceived instructiveness of the feedback. Data were analysed with multilevel techniques.

RESULTS

The perceived instructiveness of feedback provided by specialists and residents did not differ significantly. However, both were perceived to be more instructive than feedback from nursing and paramedical staff (beta(specialists) = 0.862, p < 0.01; beta(residents) = 0.853, p < 0.01). Feedback on behaviour that had been directly observed was reported to be more instructive than feedback on behaviour that had not been observed (beta(observed) = 0.314, p < 0.001). Feedback which stemmed from student initiative or a joint initiative was experienced to be more instructive than feedback which ensued from the supervisor's initiative (beta(student) = 0.441, p < 0.01; beta(joint) = 0.392, p < 0.01).

CONCLUSIONS

The expectations concerning the influence of observation and student initiative on the instructiveness of feedback were confirmed in this empirical study. Expected differences in instructiveness between feedback from specialists and residents were not confirmed.

A randomized trial comparing 2 outpatient educational settings for clerkship students: private practice office vs hospital clinic

OBJECTIVE

The purpose of this study was to compare medical students' experiences in 2 outpatient educational settings.

STUDY DESIGN

For their 2-week ambulatory rotation, all 76 medical students were randomized to either a hospital clinic or a private office. Seventy students completed the survey.

RESULTS

When compared to students working in a private office setting, hospital clinic students reported more breast exams, speculum exams, uterine fundal height measurements, wet mount reading, contraceptive counseling, and first obstetrical and gynecologic patient visits. Students in the hospital clinic were more likely to report having educationally valuable opportunities to interview and examine patients and to discuss the differential diagnosis with their preceptor. Nevertheless, medical students randomized to the private office were more likely to report considering obstetrics and gynecology as a career.

CONCLUSION

The hospital clinic provided students with more clinical experiences. Medical students assigned to a private office, however, were more likely to consider obstetrics and gynecology as a career.

What makes a good clinical teacher in medicine? A review of the literature

PURPOSE

The authors perform a review of the literature pertinent to the question, "What makes a good clinical teacher in medicine?"

METHOD

After framing the question, based on discussions of their own experiences with clinical teachers, the authors performed a search of the literature pertinent to the question, "What are the qualities of a good clinical teacher in medicine?" Between July and December, 2006, they reviewed titles from Index Medicus (1909-1966), PubMed (1966 to the present), PubMed Related Articles, and referenced articles. The initial selections were chosen by scanning pre-1966 Index Medicus title lists and post-1966 abstracts. Chosen articles were then read in their entirety, and those which described specific characteristics of clinical teachers were selected for inclusion. Qualitative analysis was used to identify themes.

RESULTS

From 4,914 titles, 68 articles were selected for analysis-26 published before 1966, and 42 published after 1966. Four hundred eighty descriptors were identified and grouped into 49 themes, which were clustered into three main categories: physician, teacher, and human characteristics. Echoing the authors' intuitive descriptions, noncognitive characteristics dominated the descriptions and themes.

CONCLUSIONS

Excellent clinical teaching, although multifactorial, transcends ordinary teaching and is characterized by inspiring, supporting, actively involving, and communicating with students. Faculty development programs and future research should focus on development of the noncognitive attributes of clinical teachers, as well as the knowledge and skills associated with effective teaching.

Examination of a clinical teaching effectiveness instrument used for summative faculty assessment

This study explores whether a clinical teaching effectiveness (CTE) instrument provides valid scores for summative faculty assessment. The sample included all CTE instruments (n = 10,087) that learners (N = 1,194) completed to assess clinical teachers (N = 872) during 1 academic year. The authors investigated response processes (e.g., missing data, straight-line responses, level of learner), internal structure (e.g., confirmatory and exploratory factor analysis), teaching ratings by learner group (medical student or resident), and relation to other variables (e.g., correlation with global rating). Response processes identified a high prevalence of straight-line responses (same rating across all items) and differential patterns of missing data by learner group. Medical students rated their teachers higher than residents, and CTE scores had different factor structures depending on learner group. High correlation coefficients of CTE items with a single rating of overall teaching performance suggest that learners consider global performance when assessing clinical teaching performance.

A developmental approach to evaluating competence in clinical reasoning

In the past two decades there has been tremendous worldwide interest in assessing the clinical competence of learners in medical education. This interest marks a philosophical shift toward greater objectivity, accountability, and predictive power in the evaluation of trainees. One of the core competencies in medical education is clinical reasoning. Because veterinary and human medical training share several similarities and differences, a review of the current state of clinical reasoning competency assessment in medical education may be useful for veterinary educators. This article covers the core competency of clinical reasoning (not other important competencies, such as physical examination or communication) and reviews research from medical education on the development of clinical reasoning and its implications for the transition from novice to expert. Four common stage-related learner difficulties are described: reduced knowledge, dispersed knowledge, tunnel vision, and the outsider. Specific approaches to measuring competence in clinical reasoning for each developmental level are recommended. Finally, two specific examples of evaluation based on a developmental approach to clinical expertise, the RIME (reporter, interpreter, manager, expert) system and the Script Concordance Test (SCT) methods, are discussed.

Students' perception of the characteristics of effective bedside teachers

BACKGROUND AND METHODS

To determine a student perspective of the characteristics of ideal bedside teachers, a 25-item questionnaire was administered to 84 final-year medical students. The items were constructed to check for two domains of 'Communication' and of 'Demographics'. The former included behaviours such as providing constructive feedback, respecting patient confidentiality and encouraging critical thinking, while the latter included characteristics such as gender, academic rank and language skills.

RESULTS

The students identified the characteristics in the 'Communication' domain as being far more important determinants of ideal bedside teaching than the 'Demographics' domain. Factor analysis showed that of the questions designed to determine communication all but one loaded unequivocally into a single factor, while the demographics were best described by two additional factors. Both these factors represented teacher properties that were difficult or impossible for the teacher to modify, while those in the communication domain were all amenable to change.

CONCLUSIONS

These results are consistent with data from the literature on the broader aspects of clinical teaching, and imply that the ideal bedside teaching experience from the perspective of the students is heavily influenced by teacher behaviours than that can be modified.

The structure of program evaluation: an approach for evaluating a course, clerkship, or components of a residency or fellowship training program

BACKGROUND

Directors of courses, clerkships, residencies, and fellowships are responsible not only for determining whether individual trainees have met educational goals but also for ensuring the quality of the training program itself. The purpose of this article is to discuss a framework for program evaluation that has sufficient rigor to satisfy accreditation requirements yet is flexible and responsive to the uniqueness of individual educational programs.

SUMMARY

We discuss key aspects of program evaluation to include cardinal definitions, measurements, needed resources, and analyses of qualitative and quantitative data. We propose a three-phase framework for data collection (Before, During, and After) that can be used across undergraduate, graduate, and continuing medical education.

CONCLUSIONS

This Before, During, and After model is a feasible and practical approach that is sufficiently rigorous to allow for conclusions that can lead to action. It can be readily implemented for new and existing medical education programs.

A shared professional framework for anatomy and clinical clerkships

Threats to the professionalism of medical practice in the United States have resulted in an intense focus by educational organizations on what professionalism is, on how to define it, and how to evaluate it. This essay discusses alternative educational frameworks in which professionalism can be located. As the traditional analytic framework (knowledge, skills, and attitudes) and developmental frameworks are more familiar, emphasis will be placed on a "synthetic" framework that expresses a student's progress as "reporter," "interpreter," and "manager/educator." This "RIME" framework attempts to capture the classic rhythm of observation-reflection-action that is familiar to all scientists and clinicians, and attempts to express in less generic, more behavioral terms how skills, knowledge, and attitudes must all be brought to bear at the same time by a successful student. It is argued that the complexity of professional development can be embraced with simplicity, without being simplistic.

Study of Selected Outcomes of Medical Students Who Fail USMLE Step 1

PURPOSE

Passage of the United States Medical Licensing Exam (USMLE) is required to obtain a medical license in the United States. Currently the majority of US medical schools require passage of USMLE Step 1 for either promotion to the third year or graduation from medical school. Virginia Commonwealth University School of Medicine (VCUSOM) requires that students take the USMLE but does not require passing of USMLE Step 1 for promotion or graduation. This policy enabled the authors to analyze performance outcomes during clinical rotations and monitor the residency match for a group of students who failed USMLE Step 1 on the first attempt.

METHODS

Third year clerkship grades and residency match results were reviewed for 64 students of the graduating classes of 1999-2005 who failed Step 1 on the first attempt. An equal number of students who passed Step 1 were randomly selected from each class as a comparison group. Average clinical performance ratings, NBME subject exam scores and final third year clerkship grades for the two groups were compared. Residency match rates and specialty certification were also compared.

RESULTS

The USMLE Fail Group had more Pass and fewer Honors clerkship grades than the comparison group. Subject exam scores were significantly lower in the USMLE Fail Group in all clerkships. Clinical performance ratings were significantly lower in the Fail group in three out of six clerkships: Internal Medicine, Pediatrics, and Psychiatry. However, 82% of the USMLE Fail Group later passed USMLE Step 1 and 2. Fifty-nine of the 64 students in the USMLE Fail Group matched for a residency, whereas all of the students in the Pass Group matched for a residency.

CONCLUSION

Students who fail USMLE Step 1 have lower final clerkship grades due in part to lower NBME subject exam scores. The majority of these students, however, successfully pass USMLE Step 1 prior to graduation, go on to graduate medical training, and become board certified in their specialty.

The impact of resident duty hours reform on the internal medicine core clerkship: results from the clerkship directors in internal medicine survey

PURPOSE

In July 2003, resident duty hours regulations were implemented. The impact of these regulations on medical student education has received minimal attention. The objective of this study was to evaluate the perceptions of internal medicine clerkship directors about the impact of resident physician duty hours reform on medical student teaching, assessment, and clerkship structure.

METHOD

A survey was sent to 114 institutional members of Clerkship Directors in Internal Medicine in May 2004. The survey included 17 attitude items rated on a 5-point Likert scale, five items related to clerkship structure, and four open-ended questions. Descriptive statistics were performed on the responses.

RESULTS

Ninety-six surveys were returned (84%). The majority of respondents did not believe duty hours reform had a positive impact on clerkship students' educational experiences, whereas 48.3% agreed or strongly agreed that residents had more difficulty evaluating students' clinical skills. There was not a significant change in inpatient clerkship structure after duty hours implementation. Time for teaching students, concerns about a shift-work mentality, and student continuity with their teams were major challenges. Impact on ambulatory internal medicine rotations was minimal.

CONCLUSIONS

Internal medicine clerkship directors are concerned about the impact of resident duty hours reform on student education. Additional studies of this educational impact are needed.

Learning Activities and Third-Year Medical Student Ratings of High Quality Teaching Across Different Clerkships

OBJECTIVE

To identify and compare learning activities that students associate with high quality teaching across clerkships.

METHODS

For six months, 110 third year medical students recorded data on learning activities and teaching quality using personal digital assistants (PDAs) during five different required clinical clerkships. Univariate and multivariate analyses were performed to assess the association between learning activities and student ratings of high teaching quality.

RESULTS

11,450 teaching interactions were recorded. Univariate analysis revealed that feedback was associated with perceptions of high quality teaching in all clerkships. Proposing a plan, formulating an assessment and giving an oral case presentation were associated with high quality teaching in 80% of the clerkships (p < 0.01). Multivariate analysis demonstrated that receiving high quality feedback was an independent predictor of student ratings of high quality teaching for all clerkships.

CONCLUSION

Receiving high quality feedback is the learning activity most strongly associated with students' ratings of high quality teaching across four different clerkships.

Medical student evaluation of teaching quality between obstetrics and gynecology residents and faculty as clinical preceptors in ambulatory gynecology

OBJECTIVE

This study was undertaken to compare teaching quality between obstetrics/gynecology resident and faculty preceptors in ambulatory gynecology as determined by medical student evaluation.

STUDY DESIGN

A prospective assessment of medical student evaluations of resident and faculty preceptors in ambulatory gynecology was conducted at Brigham and Women's Hospital. Forty-eight third-year medical students evaluated preceptors on 15 teaching quality items and recorded their clinical experience during their obstetrics/gynecology clerkship from July 2004 to June 2005 for a total of 138 evaluations.

RESULTS

Faculty scores were statistically higher than resident scores on 4 of the 15 teaching quality items; acting as an appropriate clinical role model, emphasizing evidence-based learning, being enthusiastic about teaching and patient care (P < .05). Students performed significantly more Papanicolaou tests and pelvic and breast examinations during ambulatory sessions with residents than with faculty (P < .001).

CONCLUSION

Residents and faculty contribute important and different aspects of teaching experiences for medical students in ambulatory gynecology.

Bedside teaching in the emergency department

Bedside teaching is a valuable instructional method that facilitates the development of history and physical examination skills, the modeling of professional behaviors, and the direct observation of learners. The emergency department (ED) is an ideal environment for the practice of bedside teaching, because its high patient volume, increased acuity of illness, and variety of pathology provide plentiful patient-centered teaching opportunities. Unfortunately, the pressures of ED overcrowding at many institutions now limit the available time for formal bedside teaching per patient. This article will discuss the historical decline of bedside teaching on the wards, address obstacles to its use in the ED, and reestablish its specific benefits as a unique educational tool. The authors propose several practical strategies to increase bedside teaching by academic emergency physicians (EPs). These techniques emphasize careful preparation and a focused teaching approach to overcome the inherent challenges of a typically busy ED shift.

Student evaluation of the clinical 'curriculum in action'

AIM

To examine how students' evaluations of the environment, process and outcome of clinical learning interrelated and correlated with assessment results.

METHOD

A post hoc study in the 3rd of 5 years in a student-centred, horizontally integrated, objective-based medical curriculum. In the last week of each module, students evaluated what they had learned and how they had learned it using a previously validated, web-based scale. The interrelationships between scale variables and their relationships with summative assessment results were tested using factor analysis, correlation analysis and stepwise multiple regression analysis.

RESULTS

Student evaluation yielded 4 summary measures: 2 reflected learning outcomes ('real patient learning' and 'curriculum coverage'), 1 reflected process ('quality of instruction') and 1 reflected environment ('conditions for learning'). They fitted a causal model according to which instruction, conditions for learning and curriculum coverage favoured real patient learning. Real patient learning was rated higher in women than men, and the measures were associated more strongly in women. Performance in end-of-year summative assessments was predicted strongly by mid-year performance but by no other measure.

CONCLUSIONS

Students' evaluations of their learning environment and instructional processes correlated with their assessments of 2 outcomes of the curriculum in action: curriculum coverage and real patient learning. There was little shared variance between those measures and students' performance in summative assessments. Given its formative potential, students' evaluation of their curriculum in action could play a useful part in learner-centred clinical education. There is a possibility, which needs further research, that women's evaluations have greater predictive validity than men's. Assessment performance should be regarded not as a solitary gold standard but as just 1 measure of educational outcome.

Does group discussion of student clerkship performance at an education committee affect an individual committee member's decisions?

BACKGROUND

To determine whether deliberation as part of a group affects an individual's decisions for grading and remediation of marginal students.

METHOD

In academic year 2001-02, members of a Department of Medical Education Committee prospectively completed pre- and postdiscussion surveys about their decision-making processes for third-year internal medicine clerkship students presented for marginal performance. Postdiscussion written comments were analyzed qualitatively.

RESULTS

A total of 23 (14%) students were discussed, resulting in 297 individual committee member decisions (3,090 educator-minutes). A total of 76 of 297 (25%) decisions were altered following committee deliberations, changing the grade and/or remediation for nine students. Only seven of 76 (9%) changes were anticipated. Qualitative analysis revealed four underlying themes for changing: influence of members; data provided; clarification of process; and factors outside the clerkship.

CONCLUSIONS

Group discussion influenced individual committee members' decisions for one-quarter of marginal students. The committee process allowed for clarification of the record, faculty development, and full discussion of student performance.

Learning/feedback activities and high-quality teaching: perceptions of third-year medical students during an inpatient rotation

PURPOSE

To identify specific learning activities (and teaching methods) that students associate with high-quality teaching in the inpatient setting.

METHOD

For ten months in 2003-04, 170 third-year medical students recorded data on learning/feedback activities and teaching quality via personal digital assistants during the inpatient portion of a required two-month medicine clerkship at four sites affiliated with the Medical College of Wisconsin. Univariate and multivariate analyses were performed to assess the association between learning/feedback activities and students' perceptions of high-quality teaching.

RESULTS

A total of 2,671 teaching encounters were rated by 170 students during their required inpatient medicine rotations. Bedside teaching was reported in almost two-thirds of teaching/learning encounters. Feedback on case presentation and differential diagnosis were the inpatient feedback activities most often provided by faculty. The univariate analysis revealed that students' perceptions of high-quality teaching was associated with receiving mini-lectures, developing short presentations on relevant inpatient topics, bedside teaching, case-based conferences, learning electrocardiogram and chest X-ray interpretation, teaching with other team members present (p <. 001), and receiving feedback on history and physical examination, on case presentation, at the bedside, on differential diagnosis, and on daily progress notes. Results from the regression analysis revealed that giving mini-lectures on inpatient topics, teaching electrocardiogram and chest X-ray interpretation, providing feedback on case presentation, and at the bedside were predictors of overall high-quality teaching.

CONCLUSIONS

Aspects of feedback, giving mini-lectures, and learning test-interpretation skills were the learning and feedback activities associated with students' perceptions of high-quality teaching. In an increasingly time-pressured inpatient environment, clinical educators should understand which activities students value.

Effect of a physical examination teaching program on the behavior of medical residents

CONTEXT

The reliance on physical examination as a diagnostic aid is in decline.

OBJECTIVE

To determine whether an educational program can increase the use of physical examination by medical residents.

DESIGN AND PARTICIPANTS

A series of educational workshops were provided to 47 second- and third-year medical residents at a large academic teaching hospital.

MEASUREMENTS

Interns and students reported the frequency and depth of clinical examination performance on morning rounds by their residents before and up to six months after the workshops. Behavior before and after the workshops was compared using a mixed model.

RESULTS

A total of 374 reports were returned (77% response). After adjusting for the type of service and observer, there was a statistically significant 23% increase (P=.02) in the performance of physical examination among residents who attended the course. Residents significantly increased the fraction of patients they examined on rounds (absolute increase 11%, P=.002) but did not increase the depth of their examination. The change was greatest on general medical teams, among whom the performance of physical examination had been least frequent. Teaching and feedback events on medicine teams by residents to their interns (2.8 and 1.1 events per 2 weeks, respectively) and medical students (5.9 and 2.8 events per 2 weeks, respectively) remained infrequent.

CONCLUSIONS

A skills improvement program can significantly increase the frequency of physical examination, but teaching and feedback events remain sporadic and infrequent.

Medical students' perceptions of the elements of effective inpatient teaching by attending physicians and housestaff

BACKGROUND

Most studies of effective inpatient teaching have focused on teaching by attending physicians.

OBJECTIVE

To identify and compare medical students' perceptions of behaviors associated with teaching effectiveness of attending physicians and housestaff (residents and interns).

DESIGN AND PARTICIPANTS

Third-year students who spent 4 weeks on a general internal medicine inpatient service during academic year 2003-2004 completed surveys using a 5-point Likert-type scale. Students evaluated numerous teaching behaviors of attendings and housestaff and then evaluated their overall teaching effectiveness.

MEASUREMENTS

Each behavior was correlated with the perceived teaching effectiveness in univariate and regression analyses.

RESULTS

Seventy-two students were taught by 23 attendings and 73 housestaff. Of 144 possible teaching evaluations, they completed 142 (98.6%) for attendings and 128 (88.9%) for housestaff. The mean rating for perceived teaching effectiveness was 4.48 (SD 0.82) for attendings and 4.39 (SD 0.80) for housestaff. For attending physicians, teaching effectiveness correlated most strongly with enthusiasm for teaching (R(2)=63.6%) but was also associated with inspiring confidence in knowledge and skills, providing feedback, and encouraging students to accept increasing responsibility. Housestaff teaching effectiveness correlated most strongly with providing a role model (R(2)=61.8%) but was also associated with being available to students, performing effective patient education, inspiring confidence in knowledge and skills, and showing enthusiasm for teaching. Regression models explained 79.7% and 73.6% of the variance in evaluations of attendings and housestaff, respectively.

CONCLUSIONS

Students' perceptions of effective teaching behaviors differ for attending physicians and housestaff, possibly reflecting differences in teaching roles or methods.

Assessing the impact of a comprehensive sexually transmitted disease curriculum on learning outcomes

OBJECTIVE

Our purpose was to assess the impact of a curriculum designed to improve third-year medical students' knowledge of sexually transmitted diseases, measured by sexually transmitted disease-related items from the National Board of Medical Examiners subject examination and by a locally developed sexually transmitted disease test.

STUDY DESIGN

All students (n = 108) were exposed to a new sexually transmitted disease curriculum: a 2-hour laboratory module, lectures, syllabus, and locally developed pretest/posttest with review of the test prior to taking the National Board of Medical Examiners subject examination. Students were randomized to a attend sexually transmitted disease clinic (n = 47) versus no sexually transmitted disease clinic (n = 61).

RESULTS

Students performed equal to or better than the national average on 85% of the National Board of Medical Examiners sexually transmitted disease-related items after curriculum institution, compared with 56% of the test items prior ( P < .001). Magnitude of improvement was dependent on clerkship timing, with greater improvement in students taking the obstetrics-gynecology clerkship earlier in the third year. Mean postcurriculum test scores of sexually transmitted disease improved significantly ( P < .001), independent of clinic site and clerkship timing.

CONCLUSION

The curriculum for sexually transmitted disease produced significant improvement in third-year medical students' knowledge of sexually transmitted disease. This might have an impact on future prevention and control of sexually transmitted diseases in communities in which these students practice.

Is a faculty developed pretest equivalent to pre-third year GPA or USMLE step 1 as a predictor of third-year internal medicine clerkship outcomes?

BACKGROUND

Preclinical grade point average (GPA), and United States Medical Licensing Examination (USMLE) Step 1 are well-known preclerkship methods of identifying students at risk of poor performance. These measures are not available at all medical schools and may be considered prejudicial.

PURPOSE

Does an examination on the first day of a clerkship (pretest) correlate with grade point average (GPA) or USMLE Step 1 scores, and does it provide equivalent insight in predicting clerkship outcomes?

METHODS

At this medical school, students take a faculty-developed pretest on the first day of the internal medicine clerkship. At the end of the clerkship, teachers recommend "clinical points," and students take the National Board of Medical Examiners (NBME) subject test. Between 1997 and 2001, data from 585 students (88%) were available.

RESULTS

Correlations between the faculty-developed pretest, GPA, and USMLE Step 1 scores were good, r = 0.57 to 0.66; disattenuated, r = 0.70 to 0.85. All variables were similar significant predictors of NBME subject test and clinical point outcomes.

CONCLUSIONS

A faculty-developed pretest given on the 1st day of the internal medicine clerkship can identify students at risk of poor performance and may offer advantages over USMLE Step 1 and preclinical GPA.

Intersite consistency as a measurement of programmatic evaluation in a medicine clerkship with multiple, geographically separated sites

PURPOSE

This study introduces "intersite consistency" as a measurement of programmatic evaluation and demonstrates its feasibility and construct validity.

METHOD

Student data in our multisite, geographically separated clerkship were collected prospectively over a ten-year period (1990-2000). We calculated mean scores for each clerkship measurement and analyzed these data on both a yearly and a 10-year cumulative basis. Analyses of variance (ANOVA) and linear regression were used for statistical analysis.

RESULTS

Data for 1,632 (98%) students were included in our study. During this ten-year study period, we had 22 different on-site clerkship directors at seven clerkship sites. ANOVA and linear regression of year-to-year and cumulative data did not demonstrate an effect of site on student outcomes.

CONCLUSIONS

Intersite consistency can be used as one measure of programmatic evaluation for multisite clerkships.

The R-I-M-E method for evaluation of medical students on an obstetrics and gynecology clerkship

OBJECTIVE

The purpose of this study was to implement and assess the R-I-M-E (Reporter, Interpreter, Manager, Educator) system as a method for evaluation for medical students during the obstetrics and gynecology core clerkship.

STUDY DESIGN

With the R-I-M-E method (which is an objective framework for evaluation), mid clerkship review sessions are held during which residents verbally assess each student on competencies that include problem identification and reasoning skills, communication skills, physical examination skills, written documentation, and professional attributes. The clerkship director provides timely feedback to the students. Residents and students were surveyed to assess their attitudes about the R-I-M-E method.

RESULTS

Most of the attitudes of residents and students were positive about the R-I-M-E evaluation system. The directors feel that R-I-M-E method allows more meaningful, complete, and timely feedback to students, especially in areas that are traditionally difficult to assess, such as professionalism.

CONCLUSIONS

The R-I-M-E method is an effective evaluation tool for medical students during an obstetrics and gynecology clerkship and is well received by teachers and learners.

What medical students value most during their clinical clerkships from department chairs

OBJECTIVE

The purpose of this study was to evaluate the qualities that medical students value most in department chairs during clinical clerkships.

STUDY DESIGN

All fourth-year students were asked to complete an anonymous survey voluntarily. The students ranked nine qualities of a chair using a visual analog scale that ranged from "strongly disagree" to "strongly agree." Also assessed was the student's level of contact with each department chair during the six core clerkships.

RESULTS

Sixty-seven of 73 students (91.8%) completed the survey. Almost two thirds of the students (63%) had minimal or no contact with clinical chairs. The most desirable qualities of the department chair included advocating teaching by the residents and faculty, meeting with students (if possible, at the beginning, middle, and end of the clerkship), and participating in teaching in the classroom or clinic settings. Least important qualities involved knowing the student as a person and displaying an air of authority. The most common theme of written comments was that the department chair sets the tone for the department by advocating teaching and by encouraging respect for students.

CONCLUSION

Medical students want clinical chairs to be visibly proactive toward teaching, meet with them periodically, and participate in teaching.