Faculty development effects on clerkship grades

A systematic review of faculty development programs based on the Harden teacher's role framework model

BACKGROUND

Despite the changing roles of faculty in the health professions over the past two decades, none of the reviews has been paid enough attention to the impact of the faculty development programs on these roles. The objective of this review is to synthesize the existing evidence that addresses the questions: "What are the types and outcomes of faculty development programs based on the Harden teachers' role framework and which of the areas described by Harden and Crosby are the authors referring to?"

METHODS

This review was conducted according to the guidance for Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) framework. In 2020, a literature search was conducted in MEDLINE/PubMed, Scopus, ERIC, ScienceDirect, Google Scholar, Magiran and SID databases. The review included 119 studies (between 1990 and 2020) that met the review criteria. Data were extracted using a modified coding sheet. We used the modified Kirkpatrick model to assess the educational outcomes of faculty development programs.

RESULTS

The majority of faculty development programs were workshops (33.61%) with various durations. Most programs focused on the domain of information provider and coach (76.47%), followed by the facilitator of learning and mentor (53.78%) and assessor and diagnostician (37.81%). Only five faculty development programs focused on the domain of role model. The majority (83.19%) of outcomes reported were at level 2B, level 1 (73.95%) and level 2A (71.42%). Gains in knowledge and skills related to teaching methods and student assessment were frequently noted. Behavior changes included enhanced teaching performance, development of new educational curricula and programs, improved feedback and evaluation processes, new leadership positions, increased academic output and career development. The impact on the organizational practice continued to be underexplored.

CONCLUSION

Based on the review findings, broadening the scope of faculty development programs beyond the traditional roles of the faculty members by utilizing a competency-based framework for developing a comprehensive faculty development program is recommended. Attention to individualized form of faculty development programs and incorporating more informal approaches into the design and delivery of faculty development programs is also needed.

Building reliable and generalizable clerkship competency assessments: Impact of 'hawk-dove' correction

PURPOSE

Systematic differences among raters' approaches to student assessment may result in leniency or stringency of assessment scores. This study examines the generalizability of medical student workplace-based competency assessments including the impact of rater-adjusted scores for leniency and stringency.

METHODS

Data were collected from summative clerkship assessments completed for 204 students during 2017-2018 the clerkship at a single institution. Generalizability theory was used to explore variance attributed to different facets (rater, learner, item, and competency domain) through three unbalanced random-effects models by clerkship including applying assessor stringency-leniency adjustments.

RESULTS

In the original assessments, only 4-8% of the variance was attributed to the student with the remainder being rater variance and error. Aggregating items to create a composite score increased variability attributable to the student (5-13% of variance). Applying a stringency-leniency ('hawk-dove') correction substantially increased the variance attributed to the student (14.8-17.8%) and reliability. Controlling for assessor leniency/stringency reduced measurement error, decreasing the number of assessments required for generalizability from 16-50 to 11-14.

CONCLUSIONS

Similar to prior research, most of the variance in competency assessment scores was attributable to raters, with only a small proportion attributed to the student. Making stringency-leniency corrections using rater-adjusted scores improved the psychometric characteristics of assessment scores.