Pharmacy Student Perceptions and Preferences of In-person Versus Video-Recorded Evaluations in Skills-Based Courses

Transition from in-person to telehealth standardized patient encounters: A retrospective analysis of 18 cases

INTRODUCTION

Due to the COVID-19 pandemic, many pedagogical activities shifted from in-person to virtual modalities, including standardized patient (SP) encounters. While many academic institutions may not have previously been conducting SP encounters virtually, this change provided student pharmacists with new opportunities to learn and practice telehealth techniques, which are important with the continuously evolving healthcare landscape.

METHODS

This retrospective study analyzed pharmacy student performance on and perceptions of SP cases in fall 2019 (in-person) and fall 2020 (telehealth) semesters. The primary objective was to compare average total scores on SP cases from fall 2019 (in-person) and fall 2020 (telehealth) for first-, second-, and third-year pharmacy students (P1, P2, and P3, respectively). Secondary objectives were to compare average scores from case-specific and relationship and communication (R&C) checklists, as well as to describe student perceptions of the pedagogy.

RESULTS

Total scores on SP cases decreased from in-person to telehealth for the average of all P1 SP cases, but not for P2 or P3 cases. Case-specific scores did not change, but R&C scores decreased significantly across all three years. Student survey data revealed difficulty with communicating effectively and building rapport via telehealth.

CONCLUSIONS

Overall, student performance on in-person vs. telehealth SP encounters was similar, though P1 students performed more poorly on SP cases conducted via telehealth compared to P2 or P3 students. Thus, faculty could consider focusing on in-person encounters for early learners before introducing virtual encounters. This study also highlights the difficulty in establishing rapport when communicating with patients via telehealth.

Evaluation of a Rubric to Assess Pharmacy Student Performance in a Telehealth Simulation Exercise

Objective. To evaluate a telehealth simulation rubric for a pharmacy skills lab course using interrater reliability (IRR) to compare different evaluator types.Methods. A rubric was developed and used to score student pharmacists' competency in a telehealth simulation for a pharmacy skills lab course. Intraclass correlation coefficients (ICCs) were calculated to determine the IRR.Results. Fifty-five recorded telehealth simulation sessions were scored three times by a pool of evaluators using the rubric. The overall ICC for the rubric was 0.631 across different types of evaluators, indicating good IRR. Differences in ICCs were observed when they were calculated for type of evaluator and rubric subcomponents.Conclusion. The rubric demonstrated good IRR across multiple types of evaluators when used in a skills lab course. Additional evaluation of the rubric and continuous quality improvement is warranted to ensure that assessment becomes more consistent with continued use, regardless of the evaluator type.

The Need to Educate Pharmacy Students in Telepharmacy and Telehealth

The global pandemic has dramatically impacted how pharmacists connect with patients. Telepharmacy and telehealth technologies are being used by pharmacists in a multitude of settings to positively influence the health and wellbeing of patients. Having pharmacists that are prepared to deliver services using telepharmacy and telehealth necessitates that Doctor of Pharmacy (PharmD) students receive tele-education. Tele-education topics may include telepharmacy and telehealth, audio and visual technologies, etiquette, law, reimbursement, and privacy and confidentiality. In many cases, the pandemic has required pharmacy programs to incorporate tele-education into curriculums quickly, resulting in limited and underdeveloped experiences for students. Tele-education should be purposefully incorporated into curriculums through a combination of didactic, simulation, and experiential training. Programs should evaluate how tele-education is taught within their curriculum and share those experiences with the Academy as pharmacy graduates will likely be providing patient care services using telepharmacy and telehealth.

Viability of Virtual Skills-based Assessments Focused on Communication

Objective To evaluate faculty and student perceptions of and performance on virtual skills-based assessments focused on communication compared to in-person assessments.Methods In spring 2020, virtual skills-based assessments were conducted. After all assessments were completed, two 12-item questionnaires, one for students and one for the faculty members who conducted the assessment, were designed to assess perceptions of virtual skills-based assessments. The surveys were distributed via an online platform to second- and third-year (P2 and P3) pharmacy students and to faculty who had participated in a virtual skills-based assessment. Scores from the spring 2020 virtual skills-based assessment were compared to scores on the in-person skills-based assessment that took place in spring 2019.Results Of the 19 faculty and 279 students invited to participate, 18 (94.7%) faculty and 241 (86.4%) students responded. The majority of faculty (88.9%) and students (63.5%) perceived the virtual skills-based assessments to be effective at simulating an interaction. However, only 33.3% of faculty and 28.6% of students preferred the virtual environment. There was not a significant difference in student performance between in-person and virtual assessments for patient consultation and SOAP note skills.Conclusion Providing sufficient formative and summative feedback to pharmacy students is a challenge, particularly in the context of skills-based assessments. Students and faculty reported that the virtual assessment provided an opportunity for an appropriate assessment of student communication skills. However, a strong preference for using virtual skills-based assessments in the future was not observed.

Changes to summative skills-based assessments within the Big Ten Academic Alliance Performance-Based Assessment Collaborative (BTAA-PBAC) due to COVID-19

Background

In Spring 2020 many academic institutions transitioned to remote learning in response to the developing COVID-19 pandemic. These changes affected skills-based training, as schools of pharmacy were forced to transition traditionally in-person assessments to a remote setting. The purpose of this article is to describe the experience of pharmacy skills lab coordinators when transitioning summative skills-based assessments (SSBA).

Methods

A web-based survey instrument administered through Qualtrics^XM was sent to all institutions in the Big Ten Academic Alliance-Performance Based Assessment Collaborative. Only one member from each institution completed the survey on behalf of the institution. The survey consisted of four sections: changes made to skills evaluated; changes made to the delivery of those evaluations; challenges to and strategies used by the skills lab program when switching to remote learning; and recommendations for incorporating remote learning within future SSBAs. Survey respondents were invited to participate in an optional unstructured interview regarding survey answers.

Results

Nine of ten invited institutions responded to the survey. Of the nine respondents, three participated in the post-survey interview. Overall, 79.5% (93/117) of skills planned to be assessed were assessed with or without modification, with 8.5% (93/117) of skills canceled and 10.3% (12/117) of skills assessments postponed. The most common challenges mentioned were the lack of preparation time, inability to assess certain skills virtually, and student barriers. The most common recommendations made were to prioritize lab components and incorporate flexibility in planning and scheduling.

Discussion

The results indicate that most skills were still assessed during the Spring 2020 semester. Though the transition to remote learning was challenging and unique for each institution, common strategies and recommendations identified here provide opportunities for academics to analyze and prioritize learning objectives and to rethink how to develop and deliver SSBAs as remote assessments.

Feedback for Learning in Pharmacy Education: A Scoping Review

Feedback is an effective pedagogy aimed to create cognitive dissonance and reinforce learning as a key component of clinical training programs. Pharmacy learners receive constant feedback. However, there is limited understanding of how feedback is utilized in pharmacy education. This scoping review sought to summarize the breadth and depth of the use of feedback within pharmacy education and identify areas for future research. PubMed, Embase, Scopus, and Web of Science were searched for English articles since January 2000 to identify studies related to feedback in pharmacy education. Sixty-four articles were included for analysis, stratified by moderate and major theory talk, where moderate theory talk explicitly included feedback into study design and major theory talk included feedback into both study design and analysis. Feedback was provided in Bachelor (14%), Master (15.6%), Doctor of Pharmacy (67.2%) and post-graduate programs (4.7%) on a variety of curricular objectives including communication and patient work up in didactic, objective structured clinical examination (OSCE), and experiential settings, and career/interview preparation in the co-curriculum. Feedback comments were mostly written in didactic courses, and both written and verbal in OSCE, experiential, and co-curricular settings. The pharmacy education feedback literature lacks depth beyond student perceptions, especially with respect to assessing the effectiveness and quality of feedback for learning. While feedback has been utilized throughout pharmacy education across myriad outcomes, several areas for inquiry exist which can inform the design of faculty and preceptor development programs, ensuring provision of effective, quality feedback to pharmacy learners.