Validity and reliability of an application review process using dedicated reviewers in one stage of a multi-stage admissions model

The Impact of Behavioral Anchors in the Assessment of Fellowship Applicants: Reducing Rater Biases

INTRODUCTION

No standardized evaluation tool for fellowship applicant assessment exists. Assessment tools are subject to biases and scoring tendencies which can skew scores and impact rankings. We aimed to develop and evaluate an objective assessment tool for fellowship applicants.

METHODS

We detected rater effects in our numerically scaled assessment tool (NST), which consisted of 10 domains rated from 0 to 9. We evaluated each domain, consolidated redundant categories, and removed subjective categories. For 7 remaining domains, we described each quality and developed a question with a behaviorally-anchored rating scale (BARS). Applicants were rated by 6 attendings. Ratings from the NST in 2018 were compared with the BARS from 2020 for distribution of data, skewness, and inter-rater reliability.

RESULTS

Thirty-four applicants were evaluated with the NST and 38 with the BARS. Demographics were similar between groups. The median score on the NST was 8 out of 9; scores <5 were used in less than 1% of all evaluations. Distribution of data was improved in the BARS tool. In the NST, scores from 6 of 10 domains demonstrated moderate skewness and 3 high skewness. Three of the 7 domains in the BARS showed moderate skewness and none had high skewness. Two of 10 domains in the NST vs 5 of 7 domains in the BARS achieved good inter-rater reliability.

CONCLUSION

Replacing a standard numeric scale with a BARS normalized the distribution of data, reduced skewness, and enhanced inter-rater reliability in our evaluation tool. This provides some validity evidence for improved applicant assessment and ranking.

Content, costs, and characteristics of United States prepharmacy curricula

INTRODUCTION

The objective of this study was to examine pharmacy prerequisites and estimate the costs prospective students may incur to complete those requirements.

METHODS

Prepharmacy requirements for doctor of pharmacy (PharmD) degree programs in the United States (US) (n = 137) were sourced from school websites in 2018. Credit hour costs for community colleges, public four-year institutions, and private four-year institutions were collected. Chi-square and independent t-tests compared group differences.

RESULTS

Schools required 66.12 ± 8.15 prerequisite credit hours (range 41 to 91  hours). All schools required one course in general chemistry and organic chemistry. A higher proportion of schools in 2018 required anatomy/physiology and statistics when compared to 2009. Estimated costs to complete prerequisites ranged from US $16,359 at a community college to US $187,800 for a bachelor's degree at a private institution.

CONCLUSIONS

Trends in healthcare and education make it timely for schools to reconsider prerequisites. Renewed consideration should be given to identifying what is core to the profession and the minimum competencies students must demonstrate for entry into PharmD programs.