A Situational Judgment Test to Assess Students' Achievement of Accreditation Council for Pharmacy Education Standards 3 and 4

Integrated curriculum in the United States pharmacy programs

INTRODUCTION

In the last decade, significant changes in pharmaceutical sciences have influenced the delivery of pharmacy education in Pharmacy programs. Integrated curriculum is one such method considered. We aimed to describe the perceived level of integrated curriculum among PharmD programs in the US.

METHODS

From October 26th, 2021, until January 18th, 2022, faculty administrators across 138 US pharmacy colleges were surveyed. Data was collected regarding each program's perceived curriculum integration and assessment integration. Characteristics of each college, including region and the type of school (public/private), were obtained from the PharmCAS website. Programs were categorized into high-integration and low-integration groups for analysis purposes. Descriptive and comparative analysis by the level of curriculum integration was performed.

RESULTS

Overall, 60 colleges completed surveys (participation rate = 43.48%). Most schools were from the South region (38.33%) and public colleges (53.33%). The average perceived curriculum integration was 45% (SD = 23.69), while the average perceived assessment integration was 36% (SD = 25.52). Pharmacy practice [clinical sciences] (76.67%) was the most common discipline considered for integration, and the social and administrative sciences (21.67%) was the discipline least commonly considered for integration. Case-based learning (95%) was the most common pedagogy strategy to integrate knowledge from different disciplines.

CONCLUSIONS

Integrated curriculum implementation in the US PharmD programs varied across colleges. While most programs integrated their clinical practice courses, social and administrative sciences was the course least commonly integrated. Very limited progress in assessment integration was perceived.

Opening the Black Box: Agreement and Reliability of a Situational Judgment Test Across Multiple Institutions

OBJECTIVE

The primary objective of this study was to examine the levels of agreement and reliability of a situational judgment test (SJT) using a diverse pool of pharmacy practice faculty as subject matter experts. Secondary aims included analyses to build support for test validity and fairness.

METHODS

An SJT containing 18 scenarios and 118 responses assessing empathy, integrity, and teamwork was developed and delivered to pharmacy practice faculty at 5 schools of pharmacy across the United States. Reliability was assessed by examining internal consistency, inter-rater reliability, and split-half reliability. Only responses which attained an inter-rater agreement>0.7 were included in the final version of the SJT. All responses were scored using a near-miss system, allowing higher scores for answers more closely aligned with the key, which was determined by the faculty who completed the SJT. Test fairness was reported using descriptive statistics.

RESULTS

Thirty-nine faculty across the 5 participating institutions completed the SJT. The final version of the SJT included 105 responses, achieving an inter-rater agreement of>0.7 (inter-rater reliability of 0.98). Split-half reliability was 0.72. The average score was 85.7%, and no differences in performance were observed based on demographic characteristics.

CONCLUSION

An SJT designed to assess empathy, integrity, and teamwork achieved reasonable levels of reliability among pharmacy practice faculty across the United States, and the results provided initial support for test validity and fairness. These results support a pilot to assess this SJT among students representing multiple institutions.

Perspectives of Professionalism, Self-Awareness, Leadership, and Innovation and Entrepreneurship From Experiential Education Preceptors

OBJECTIVE

The objectives of this study were to (1) quantify the perceived importance of and comfort with the Accreditation Council for Pharmacy Education Standard 4 (ACPE-S4) key elements, (2) describe how preceptors promote and assess key elements, and (3) describe preceptor perceptions of student key element skill demonstration.

METHODS

This study used a cross-sectional electronic survey to assess experiential education preceptors' perceptions of ACPE-S4 key elements. The survey consisted of 2 item categories, with 5 and 15 items included to collect information on preceptor demographics and ACPE-S4 key elements, respectively. Surveys were distributed to 451 introductory and advanced rotation preceptors. Frequencies and means were reported for quantitative survey items, open-ended survey responses were analyzed using a general interpretivist approach.

RESULTS

In total, 242 surveys were completed for a response rate of 54%. Most respondents identified as female (54.5%) and precepted community pharmacy rotations (28.5%). ACPE-S4 key elements, in order of most to least important, were professionalism, self-awareness, leadership, and innovation and entrepreneurship. Respondents varied in their conceptualization of key elements, with multiple activities used for skill demonstration and assessments. Qualitative analysis identified 2 themes: "Variation in preceptor fluency with S4 key elements" and "Communication as a means to demonstrate and evaluate ACPE-S4 key element skills."

CONCLUSION

Variation among preceptors in conceptualization, learning activities, and comfort in assessment for ACPE-S4 key elements highlights the need for support and resources to assist preceptors in developing activities and assessments for skills reflective of ACPE-S4 key elements. Preceptors commonly relied on vulnerable communication with students to assess key element skills, identifying a potential target for assessment and preceptor development. Pharmacy programs, in addition to the Academy, could benefit from using assessments already offered by preceptors. Finally, explicit evaluation criteria may help to guide skill acquisition and facilitate consistent assessment of key element skills.

Evolution of a Cocurricular Program to Support Student Professional Identity Formation

OBJECTIVE

Describe the development, implementation, and assessment of a cocurricular program intentionally designed to foster professional identity formation at a private institution.

METHODS

A cocurricular committee was established which developed a cocurriculum program in 3 phases. The committee utilized a gap analysis to inform the development of a continuing-education-based elective program (phase I), broadened program components and enhanced assessment (phase II), and continued to strengthen select areas of the affective domains through a second gap analysis and implement a summative assessment (phase III).

RESULTS

Over 2 academic years, the completion rates for reflections, continuing education programming, and community outreach in the most recent academic year were consistently above 80% by the final due date. Rates of the mentor-mentee meetings fell below 50%; however, this component is tracked by the faculty member and not the students. Community outreach monitoring was managed by the committee for the first time in the 2021-2022 academic year and completion rates increased from 64% to 82%. Student reflections indicated consistent progression toward practice readiness from first- to third-year pharmacy students. For the Pharmacy Affective Domain Situational Judgment test, 22% and 16% of first-year pharmacy students were flagged during the first and second year implemented, respectively, and only 8% of the third-year pharmacy students were flagged in both years.

CONCLUSION

Utilization of a cocurricular committee has been vital for the development, progression, and assessment of the cocurriculum at a single private institution.

Examining Validity for the Pharmacy Affective Domain Situational Judgment Test

Objective. With the integration of the affective domain in the pharmacy accreditation standards, it is important for programs to have methods for formatively assessing student competency in these areas. The objective of this study was to examine the validity evidence for a recently developed situational judgment test to assess the affective domain contained in the Accreditation Council for Pharmacy Education (ACPE) Standards 2016.Methods. After pilot testing items in multiple pharmacy schools, a revised 15-item situational judgment test instrument was developed measuring the affective domain as it relates to the pharmacy profession. The resulting Pharmacy Affective Domain Situational Judgment Test (PAD-S) was administered to students at three institutions and internal structure validity evidence was examined using item descriptives, Cronbach alpha, and results from a Rasch measurement model.Results. There were 559 valid responses included in the study. The items were of varying difficulty, with Rasch results confirming the hierarchy of items and suggesting that items were contributing to the measure. The Cronbach alpha was 0.70, suggesting acceptable reliability. However, the reliability items from the Rasch model were lower (0.65 and 0.66), indicating that the ranking of ability was limited, which may be due to fewer items of high difficulty.Conclusion. Overall, the PAD-S performed well as a measure of the affective domain. The PAD-S may be a useful formative instrument as part of a comprehensive assessment plan and may be less resource intensive than other measures.