Evolution of a capstone exam for third-year doctor of pharmacy students

Don't worry, be APPE: Developing and refining a skills-based assessment of student preparedness for advanced pharmacy practice experiences

BACKGROUND

The Accreditation Council for Pharmacy Education (ACPE) 2025 Standards charge pharmacy programs to assess student readiness for Advanced Pharmacy Practice Experiences (APPEs), but the method of assessment is not defined. This report describes the process of developing, implementing, and refining an APPE readiness assessment (ARA) that can be adapted to individual institutional needs and learning outcomes.

EDUCATIONAL ACTIVITY

The ARA at The University of Texas at El Paso School of Pharmacy, a Hispanic-serving institution, is comprised of two practice areas: drug information and the Pharmacist Patient Care Process, each with multiple objective structured clinical examinations (OSCEs) to mimic how students will apply these skills in required APPE rotations. The ARA has been adapted for administration both virtually and in-person. Students are required to attain a 70 % average across all five OSCEs, with an opportunity to retest on any failed areas prior to progression into the APPE year.

CRITICAL ANALYSIS OF THE EDUCATIONAL ACTIVITY

Since initial implementation in 2021, faculty experience and student performance informed changes to the assessment, rubrics, and integration of skills throughout the curriculum. The average overall ARA score was an 89.13 % in 2021 and ranged from 79.74 to 82.3 % in 2022-2024. Typically, less than 10 % of the class required re-testing, however, in 2023 16 % of the class had an ARA average less than 70 % requiring re-testing. Written skills are consistently the lowest performing area for students, potentially due to language discordance between students' preferred language and the language of the assessment.

A scoping review of advanced pharmacy practice experience readiness assessment measures across schools of pharmacy

BACKGROUND

The development and implementation of Advanced Pharmacy Practice Experience (APPE) readiness assessments has been an area of increased interest to the academy since the publication of the Accreditation Council for Pharmacy Education (ACPE) standards in 2016. This scoping literature review aims to provide an updated summary of current APPE readiness assessment practices among ACPE-accredited institutions in the United States (US).

METHODS

A literature search was conducted between 2022 and 2024 using the terms "APPE student readiness," "APPE readiness assessment," "APPE preparedness," "APPE student preparedness," "pharmacy" AND "readiness assessment", "pharmacy" AND "practice readiness," and "pharmacy" AND "student practice readiness" in Pubmed and Embase. The websites for the American Journal of Pharmaceutical Education and Currents in Pharmacy Teaching and Learning were also searched using these terms. Abstracts for all results were reviewed. Abstract only and poster presentations were excluded, as well as articles centered on non-US and/or non-ACPE-accredited institutions. Data was collected regarding the outcomes assessed, type of assessment activity, and how the assessment was implemented. Results were reviewed by a second author to ensure consistency in reporting.

RESULTS

A total of 289 unique abstracts were reviewed by author pairs for inclusion based on relevance to the review objectives. A total of 13 articles were included in the final analysis. Each institution measured different knowledge areas, abilities, and entrustable professional activities (EPAs). Most programs had a summative component, with APPE readiness being primarily assessed in the final didactic year of the pharmacy curriculum. Most programs report at least one source of validity.

IMPLICATIONS

Given the lack of specific APPE readiness assessment requirements in the ACPE Standards 2016, the different assessment methods among programs were not surprising. However, the commonalities identified can be leveraged with the release of the Curricular Outcomes and Entrustable Professional Activities (COEPA) 2022 to promote a standardized definition of APPE readiness. Future research should focus on formative assessment methods embedded throughout the didactic curriculum to identify "at-risk" students prior to a "high-stakes" summative assessment at the end of the didactic curriculum that impedes student progression to the APPE year.

A Cycle of Reinforcing Challenges and Ideas for Action in Experiential Settings

Evidence suggests that both pharmacy students and preceptors are struggling in the experiential setting. Underlying this phenomenon is a potential interconnected and cyclic set of behaviors being reinforced between students and preceptors. These behaviors can contribute to or are the result of higher levels of burnout and a decrease in the development of student clinical skills and subsequent performance on rotation. In this review, the authors investigate various challenges commonly encountered in the experiential environment. These challenges can range from an observed decrease in student engagement, motivation, and critical thinking skills to an increase in preceptor burnout and culture shifts in the clinical practice environments. These factors all ultimately impact patient care and overall student performance. For each challenge identified, strategies will be presented that can be implemented by students, preceptors, and pharmacy programs to break the cyclic pattern identified.

Entrustable professional activities-based objective structured clinical examinations in a pharmacy curriculum

BACKGROUND

The integration of Objective Structured Clinical Examinations (OSCEs) within the professional pharmacy program, contributes to assessing the readiness of pharmacy students for Advanced Pharmacy Practice Experiences (APPEs) and real-world practice.

METHODS

In a study conducted at an Accreditation Council for Pharmacy Education (ACPE)-accredited Doctor of Pharmacy professional degree program, 69 students in their second professional year (P2) were engaged in OSCEs. These comprised 3 stations: best possible medication history, patient education, and healthcare provider communication. These stations were aligned with Entrustable Professional Activities (EPAs) and Ability Statements (AS). The assessment aimed to evaluate pharmacy students' competencies in key areas such as ethical and legal behaviors, general communication skills, and interprofessional collaboration.

RESULTS

The formulation of the OSCE stations highlighted the importance of aligning the learning objectives of the different stations with EPAs and AS. The evaluation of students' ethical and legal behaviors, the interprofessional general communication, and collaboration showed average scores of 82.6%, 88.3%, 89.3%, respectively. Student performance on communication-related statements exceeded 80% in all 3 stations. A significant difference (p < 0.0001) was found between the scores of the observer and the SP evaluator in stations 1 and 2 while comparable results (p = 0.426) were shown between the observer and the HCP evaluator in station 3. Additionally, a discrepancy among the observers' assessments was detected across the 3 stations. The study shed light on challenges encountered during OSCEs implementation, including faculty involvement, resource constraints, and the necessity for consistent evaluation criteria.

CONCLUSIONS

This study highlights the importance of refining OSCEs to align with EPAs and AS, ensuring a reliable assessment of pharmacy students' clinical competencies and their preparedness for professional practice. It emphasizes the ongoing efforts needed to enhance the structure, content, and delivery of OSCEs in pharmacy education. The findings serve as a catalyst for addressing identified challenges and advancing the effectiveness of OSCEs in accurately evaluating students' clinical readiness.

A Systematic Review of APPE Readiness Assessments Utilized by US Schools of Pharmacy

OBJECTIVE

The 2016 Accreditation Council for Pharmacy Education standard 25.8 requires schools of pharmacy to assess student readiness for Advanced Pharmacy Practice Experiences (APPEs). We performed a systematic review to identify how schools of pharmacy in the United States assess student readiness for APPE rotations in accordance with Accreditation Council for Pharmacy Education accreditation guidelines.

FINDINGS

From a search of 6 databases, we identified 1053 unique studies, of which 11 were eligible for inclusion in our review. The most commonly reported assessment method was the use of a capstone course; however, these courses varied significantly from school to school in duration, resources used, and content. Regardless of the specific approach used, first-time and overall pass rates were high.

SUMMARY

We found that while most studies reported using some type of capstone course for APPE readiness assessment, there was variability in how these courses were structured and the assessment methods used within the courses. The future 2025 standards may dictate a more uniform structure for readiness assessments; however, further research is needed to identify best practices regarding the assessment of APPE readiness.