Changes in Pharmacy Students' Metacognition Through Self-Evaluation During Advanced Pharmacy Practice Experiences

Improving Self-Perceived Competencies of Second-Year Pharmacy Students Through an Introductory Medication Reconciliation Rotation

OBJECTIVE

Medication reconciliation (MedRec) is an essential health care function that is particularly relevant to pharmacists' expertise and a learning opportunity for pharmacy students. Our objective was to assess change across clinical competence, confidence, and communication skills after the completion of a MedRec rotation by second-year pharmacy students.

METHODS

A retrospective post-then-pre-survey including 29 questions was developed/delivered to students after the completion of required MedRec hours. The primary end point was the change in 3 domains via summed scores from individual questions. Cohen's difference (d) was used to determine group effect size change. The secondary end points included individual question change, perceived patient impact, and subgroup analyses.

RESULTS

Of 115 second-year pharmacy students, 81.7% (n = 94) participated in the study. Students self-reported increases on the Likert scale (0-10) of 2.49 ± 1.90 in clinical competency domain, 3.57 ± 2.13 in confidence domain , and 3.12 ± 2.15 in communication skills domain, representing statistically significant and large group effect changes across all 3. A total of 21 of the 22 individual questions had large group effect changes; 1 question (nursing communications) had a moderate group effect change. Student perception of MedRec impact on patient care (Likert scale 0-10) was positive: post-rotation score 7.39 ± 1.57.

CONCLUSION

To the best of our knowledge, this is the first larger-scale study that examines student-evaluated outcomes of a MedRec-based rotation. Students self-reported high levels of post-rotation competency across all domains; students from ethnic minorities and with less work/MedRec experience increased their lower pre-rotation scores to statistically similar post-rotation scores, compared with non-minority and more experienced peers. Further study of the model and outcomes is advised.

Do students' self-reflections of performance align with their graded performance in objective structured clinical exams?

INTRODUCTION

Self-awareness of strengths and weaknesses through self-reflection are important for life-long learning and development. The aim of this study was to assess the alignment in third-year undergraduate pharmacy students' self-reflections of their objective structured clinical exam (OSCE) performance to their actual OSCE scores and explore the most common aspects students reflected on as markers of perceived performance.

METHODS

Students completed a three-station OSCE and a written self-reflection about their performance. These reflections were coded using a latent pattern content analysis, with categories defined as "doing well (≥ 50% on exam)" and "not doing well (< 50% on exam)" and compared to their actual OSCE exam scores, to determine the degree of alignment.

RESULTS

Two hundred sixty-nine students completed the OSCE and reflection. Students had a low degree of alignment between their self-reflections and actual OSCE performance. Low alignment was overwhelmingly prevalent and significant in high-achieving students with OSCE scores of ≥90%. Most common aspects students reflected on as indicators of performance were finishing on time and communicating effectively. High-achieving students reflected on aspects such as empathy, systematic questioning, and patient teach-back as aspects of their performance.

CONCLUSIONS

Student reflections on exam performance do not align with their actual performance, particularly amongst the high-achieving students. High-achieving students were more aware of the different aspects that affected their performance. To ensure that high-achieving students are aware of their strengths, educators should provide more targeted feedback mechanisms and positive reassurances to help these students become more confident in their decision-making skills.

The Divergence between Self- and Preceptor-Assessments of Student Performance during Advanced Pharmacy Practice Experiences

(1) Objectives: A divergence in self- and preceptor-evaluations of clinical skills has been noted during Advanced Pharmacy Practice Experiences (APPEs). The goal of this study was to determine the domains of overestimation of clinical skills by students during their APPE rotations. (2) Methods: Preceptor-assigned grades for APPE rotations from 2017-2022 were analyzed to identify instances of letter grade B or lower. The self- and preceptor-evaluations of APPE rotation were compared to determine the domains of divergence in evaluation between students and preceptors. (3) Results: Between 2017 and 2022, 305 student APPE rotations were graded as B or lower (~14%) by the preceptors. A statistically significant difference was noted between self- and preceptor-assigned letter grades across all practice settings including ambulatory patient care, community pharmacy, general medicine patient care, hospital/health system pharmacy, and special population patient care APPE rotations. In addition, examining the self- and preceptor evaluation rubric for these rotations revealed a statistically significant overestimation of clinical skills by students in all 9 domains of APPE evaluation. Finally, the divergence in the rating of clinical skills between student- and preceptor evaluation was found to be highest in the domains of planning and follow-up of patient care, disease knowledge, and communication with patients. (4) Conclusions: Students who fail to exhibit exemplary practice readiness during APPEs tend to overestimate their clinical skills in all domains of APPE evaluation. The results from our study support the need for additional avenues to assist in the identification of deficits in student learning before APPEs to increase their self-awareness (metacognition).

Assessing student pharmacists' confidence and knowledge of basic diabetes self-management skills

INTRODUCTION

The significance of diabetes self-management and recent advances in diabetes medications call for healthcare professionals' preparedness in educating patients. This study assessed correlations between student pharmacists' confidence and knowledge of diabetes self-management skills.

METHODS

An online questionnaire was administered to third-year student pharmacists at one United States college of pharmacy. Students indicated their level of agreement (strongly disagree, disagree, agree, strongly agree) with nine confidence items and identified the correct multiple-choice response for nine knowledge items. Correlations between confidence and knowledge scores were assessed using simple linear regression, logistic regression, and Spearman rank (Rho) test.

RESULTS

Forty-five students completed the questionnaire. The majority agreed/strongly agreed that they were confident in their knowledge for eight of the nine confidence items. The majority correctly answered six of the nine knowledge items. No associations were identified between confidence and knowledge scores in the linear regression (beta scaled = 0.25) or Spearman's correlation test (r = 0.294). In the logistic regression model, confidence and knowledge scores were associated positively for three domains: glucose meter use (odds ratio [OR] = 4.88), glucagon-like peptide-1 receptor agonist use (OR = 4.23) and hyperglycemia treatment (OR = 3.33); confidence and knowledge scores were associated negatively with one domain: insulin syringe administration (OR = 0.41).

CONCLUSIONS

This study found no overall association between confidence and knowledge scores in third-year student pharmacists' basic diabetes self-management skills, although there were significant associations with four specific domains. Student pharmacists may require additional training to improve their diabetes self-management skills.

Self-Assessment of Therapeutic Decision-Making Skills in Pharmacy Students

Objective. The ability of pharmacy students to self-regulate and reflect on knowledge and skills is important as proficient self-assessment skills guide learning strategies and prompt students to bridge their knowledge gaps. The objectives for this study were to determine how well third-year pharmacy students self-assess, explore the rationales behind their self-assessments and determine whether there is a correlation between self-assessment accuracy and academic performance.Methods. A quasi-experimental one-group pre-/post-test design was conducted with third-year pharmacy students. Examiner grades, student self-assessment grades, comparative reports, and end-of-semester grades were collected. Students were categorized into tertiles based on academic performance for data analysis. Paired t tests, Pearson r and percentage agreements were conducted to investigate self-assessment accuracy. Correlational statistical tests were implemented to examine the relationships between self-assessment accuracy and academic performance.Results. One hundred sixty-two third-year pharmacy students were included. On average, students demonstrated poor self-evaluation skills and underestimated themselves by 4.9%. Lower performing students were generally overconfident in evaluating their performance. There was no significant correlation between students' self-assessment accuracy and academic performance on the subsequent end-of-semester examination questions.Conclusion. Overall, students tended to underestimate their academic performance. Further research on self-assessment is needed to better understand how students think about their performance, which may help to improve education methods, such as inclusion of reflective practices after case-based activities.

Prescription Renewal Request Reviews by Student Pharmacists in a Family Medicine Clinic

Prescription renewal requests were reviewed by student pharmacists on advanced pharmacy practice experiences (APPE) at a primary care and family medicine clinic. Student pharmacists reviewed requests and triaged them to the respective primary care provider (PCP), along with any recommendations to optimize the medication regimen. This study aims to assess the acceptance of these recommendations as well as the student’s perception of this activity as a learning tool. A total of 35 4th-year pharmacy students participated in this activity during APPE rotations from May 2019 to March 2021. A total of 184 recommendations were made, with 128 (70%) being accepted by PCPs. Based on a post-rotation anonymous survey, students reported high levels of agreeance that this activity had a positive impact on their education in a variety of ways. This prescription renewal request review process has been shown to have a positive impact on patient care and clinic workflow while also providing pharmacy students with a helpful educational activity.

Longitudinal evaluation of pharmacy students' metacognition in interpretation of evidence-based medicine

BACKGROUND AND PURPOSE

Metacognition is one's ability to "think about thinking" and to understand what knowledge and skills one has. Pharmacy professionals are required to have high metacognitive skills to ensure they are aware of what information they know and when to utilize resources. This longitudinal evaluation of pharmacy students' metacognitive ability was conducted via analysis of students' knowledge and confidence in evidence-based medicine (EBM) skills.

EDUCATIONAL ACTIVITY AND SETTING

A survey was administered to the graduating classes of 2017 and 2018 four times, from the beginning of the second-year drug literature evaluation course (baseline) through graduation. The surveys included seven self-confidence questions for which a corresponding knowledge question was asked. Students' metacognition was evaluated on the basis of whether the level of confidence was a "match" or a "mismatch" to their corresponding level of knowledge of each question. The proportion of matches over time was determined.

FINDINGS

Fifty-eight students (22%) completed all four surveys. The mean proportion of matches at baseline was 40% and increased over time until graduation when the proportion of matches was 74%. The proportion of matches at baseline for all seven questions were ranged from 33% to 57%. An improvement in meta-cognition was seen in 5 out 7 questions.

SUMMARY

This evaluation provides evidence that as students learned material and revisited concepts, their confidence, knowledge, and potentially metacognition regarding EBM concepts increased. However, as 26% of items measured were not a match at graduation, interventions need to be designed and tested to improve student metacognition.