Self-assessment in pharmacy and health science education and professional practice

Do students' self-reflections of performance align with their graded performance in objective structured clinical exams?

INTRODUCTION

Self-awareness of strengths and weaknesses through self-reflection are important for life-long learning and development. The aim of this study was to assess the alignment in third-year undergraduate pharmacy students' self-reflections of their objective structured clinical exam (OSCE) performance to their actual OSCE scores and explore the most common aspects students reflected on as markers of perceived performance.

METHODS

Students completed a three-station OSCE and a written self-reflection about their performance. These reflections were coded using a latent pattern content analysis, with categories defined as "doing well (≥ 50% on exam)" and "not doing well (< 50% on exam)" and compared to their actual OSCE exam scores, to determine the degree of alignment.

RESULTS

Two hundred sixty-nine students completed the OSCE and reflection. Students had a low degree of alignment between their self-reflections and actual OSCE performance. Low alignment was overwhelmingly prevalent and significant in high-achieving students with OSCE scores of ≥90%. Most common aspects students reflected on as indicators of performance were finishing on time and communicating effectively. High-achieving students reflected on aspects such as empathy, systematic questioning, and patient teach-back as aspects of their performance.

CONCLUSIONS

Student reflections on exam performance do not align with their actual performance, particularly amongst the high-achieving students. High-achieving students were more aware of the different aspects that affected their performance. To ensure that high-achieving students are aware of their strengths, educators should provide more targeted feedback mechanisms and positive reassurances to help these students become more confident in their decision-making skills.

Using artificial intelligence platforms to support student learning in physiology

The advent of artificial intelligence (AI)-powered language models presents new opportunities and challenges in education. By teaching students how to craft prompts that elicit insightful responses, faculty can scaffold activities where AI acts as a supplemental resource to amplify critical thinking and support student learning. Ongoing dialogue and iteration focused on ethical usage norms can achieve the right balance between emerging technology and foundational skills development. With care and intention, AI-assisted study tactics offer students personalized support while adhering to academic standards. While AI-powered tools provide many positive opportunities, students and faculty need to learn about and use them responsibly and ethically, not as replacements for required thinking and effort. Before implementing these AI tools for studying biology, there are several key things to discuss with students. This article outlines several ways that students can employ these tools to support better learning along with a set of guidelines for all to be wary of when implementing these in an academic setting.NEW & NOTEWORTHY Utilizing of artificial intelligence tools offers a promising new technology to support student learning. This article outlines several ways that students can employ these tools to support better learning along with a set of guidelines for all to be wary of when implementing these in an academic setting.

Development and Evaluation of an Experiential Career Planning Assignment to Train Students to Assess Organizational Fit

Determining if an employment opportunity will be a good match can feel daunting, especially for veterinary graduates entering the workforce. To ease this transition, veterinary educators traditionally have attempted to provide career support through interspersed didactic lectures on career options and the preparation of employment documents. While well intended, this approach fails to address the multiple dimensions of effective career planning or the reality that career planning is a lifelong endeavor. For a career planning teaching modality to be effective, it must address all stages of career planning and provide a framework that can be adapted throughout a career. Here we describe how a four-stage career-planning model, utilized throughout higher education, was employed to create a career planning assignment for guiding students in assessing organizational fit. We describe how student feedback was used to inform revisions, resulting in an improved educational experience as measured by students' perceptions of the utility of the assignment. Additional recommendations based on instructor reflection are provided to guide creation and implementation of future assignments. Given the growing support for professional skills training in veterinary medical education, we view incorporation of such learning activities as essential to preparing students to enter the modern veterinary workplace.

Self-Reflection and Peer-Assessments Effect on Pharmacy Students' Performance at Simulated Counselling Sessions

INTRODUCTION

Verbal communication is a vital skill for pharmacists and essential for improving patient care. The aim of this study was to explore students' perception of the impact of self-reflection and self- and peer-assessment on simulated patient counselling sessions.

METHODS

Focus groups explored student perceptions of how this course and way of learning has impacted their performance at counselling patients. Data were analysed using iterative inductive thematic analysis procedures and mapped to the self-determination theory.

RESULTS

Nine focus groups with 47 pharmacy students. We identified three main themes and ten associated subthemes. These were learning style (sub-themes gradual introduction to assessment, learning through self-reflection videos, authentic assessment, individual learning compared to group learning, and learning through observation of best practice), feedback (sub-themes inconsistent feedback, summative feedback, perception of self and relationship with peers informing peer assessment) and benefits in real life practice. These themes mapped well to self-determination theory and highlighted that additional focus may be required for benefits in real-life practice.

CONCLUSION

Students' perceptions of self-reflection and self- and peer-assessment centred on learning style, feedback, and benefits in real-life practice. Additional focus on benefits of this unit of study in real-life practice and work integrated learning on placements may further strengthen the impact of these learning activities.

Using the study cycle model to support better student learning: a faculty guide

Despite years of varied school experiences, many college students do not have adaptable study methods that allow them to collect, organize, and reorganize information to critically assess what they know and what they need help with until after a poor exam performance. Faculty are frequently called on to help students become better learners. In addition, as the tasks and complexity of the material become more challenging, students will need to adopt new study habits and adapt them to each new learning opportunity to reach their goals. This article describes an evidence-based framework that teachers can share with students that will support better studying habits leading to deeper learning.NEW & NOTEWORTHY Many college students do not have adaptable methods habits that allow them to get the most out of in-class time and structure their out-of-class study time. As a result, faculty are frequently called on to help students become better learners. This article describes an evidence-based framework focusing on learning anatomy and physiology examples that faculty can share with students to support better studying habits leading to deeper learning.

Impact of a gamified simulation on pharmacy students' self-assessed competencies

INTRODUCTION

Competency-based pharmacist education develops robust professional identities and prepares graduates for future practice to ensure optimal patient outcomes. An extended gamified simulation was developed as a capstone activity for a new Australian Bachelor of Pharmacy (BPharm) program. The simulation was designed to consolidate students' knowledge, skills, and behaviors from prior learning. This research aimed to explore whether participation in an extended gamified simulation could influence pharmacy students' perceptions of their professional competencies.

METHODS

Data were collected over three years to compare a superseded Master of Pharmacy (MPharm) program with an incoming BPharm program. Final year students were invited to self-assess their professional competencies at the start and end of their final semester of study, using a digital self-assessment tool which replicated Australia's National Competency Standards Framework for Pharmacists. Participants rated their own competency against the 26 competency standards across five domains on a five-point Likert scale (not at all competent to very competent). This provided pre- and post-data to compare the simulation (BPharm intervention) and a traditional semester (MPharm comparison), in addition to final course grades.

RESULTS

From 2016 to 2019, 85 (90.4%) of 94 intervention and 50 (83.3%) of 60 comparison students completed the self-assessment of professional competencies. Participation in the gamified simulation significantly improved students' pharmacotherapeutics grades and pre-post change scores for seven of the 26 competency standards, two of the five domains, and all domains combined of the National Competency Standards.

CONCLUSIONS

An extended, gamified simulation enhances the development of pharmacy students' self-assessed professional competencies.

Myth or Reality: Self-Assessment Is Central to Effective Curriculum in Anatomical Pathology Graduate Medical Education

Self-assessment, a personal evaluation of one’s professional attributes and abilities against a perceived norm, has frequently been cited as a necessary component of self-directed learning and the maintenance of competency within regulated health professions, including the medical professions. However, education research literature has consistently shown uninformed personal global assessment of performance to be inaccurate in a variety of contexts, and have limited value in a workplace-based curriculum. Incorporating known standards of performance with internal and external data on the performance improves a learner’s ability to accurately self-assess. Selecting content suitable for self-assessment, providing explicit assessment standards, encouraging feedback-seeking behaviors, supporting a growth mindset, and providing quality feedback in a supportive context are all strategies that can support learner self-assessment, learner engagement in reflection, and action on feedback in Anatomical Pathology graduate medical education.

Validity Evidence for a Pharmacists' Patient Care Process Self-Efficacy Scale Among Pharmacy Students

Objective. To further refine and examine the validity of an instrument for assessing pharmacy students' self-efficacy in implementing the Pharmacists' Patient Care Process (PPCP) components.Methods. An instrument was developed and pilot tested in spring 2018 at one college of pharmacy. In spring 2019, a modified version of the instrument, the PPCP Self-Efficacy Scale (PPCP-SES), was administered to third professional year (P3) pharmacy students at seven institutions. Self-efficacy items were based on Bandura's theory of self-efficacy, and students were asked to rate each item on a continuous scale (0-100). Data analyses included descriptive statistics, reliability analysis, and confirmatory factor analysis (CFA).Results. Three hundred P3 students completed the PPCP-SES. The domain-specific Cronbach alpha coefficients were: collect, α=.89; assess, α=.92; plan, α=.95; implement, α=.96; and follow-up, α=.95. Based on the factor analysis results, three items were removed. Model fit statistics indicated the overall instrument had moderate goodness of fit.Conclusion. Results indicate the PPCP-SES demonstrated initial evidence of validity for use by pharmacy faculty members to identify students' self-efficacy related to implementing components of the PPCP. Future research is needed to examine validity evidence in other student populations and among practicing pharmacists.

Preparing lifelong learners for delivering pharmaceutical care in an ever-changing world: a study of pharmacy students

BACKGROUND

Continuing professional development (CPD) continues to gain acceptance as a model for health care professionals to engage in lifelong learning (LLL). Many pharmacy schools have not adopted yet specific programs targeting the development of LLL skills, though LLL is widely accepted as an essential competence. This paper examines the effectiveness and utility of a longitudinal CPD training program.

METHODS

A CPD simulation course was introduced to a cohort of fifth year students in Northern Cyprus in the 2018-2019 academic year. The program was delivered as an interactive orientation course in one semester; meanwhile, in the second semester, the students applied the CPD cycle and completed their portfolios during their final experiential practice. A mixed-methods approach was used to evaluate the outcomes of the intervention using students' preparedness for lifelong learning (SPLLL) self-administered questionnaire delivered pre-post program, focus group sessions for students to reflect on the course experience, and instructors' evaluations of portfolios.

RESULTS

Following the implementation of the course, students' assessment scores were significantly higher overall and for all scale domains, including "knowledge, skills, attitude and practice", compared to the baseline assessment. Additionally, compared to fifth year students who responded to the second SPLLL questionnaire, the intervention group students' assessment was significantly higher in knowledge, skills, and practice. The qualitative analysis reported high student satisfaction and achievement of the course objectives. Nineteen of the students scored high on their portfolios.

CONCLUSION

The CPD simulation course provided students with opportunities to practice and develop self-assessment and self-management skills that are all desirable for lifelong learning and prepared them for CPD.

Comparing faculty evaluations of student journal club presentations with student self- and peer evaluations during advanced pharmacy practice experiences

BACKGROUND AND PURPOSE

Self-evaluation is a critical step in professional development. Peer evaluation may enhance student learning and help peer evaluators recognize their own limitations. However, these evaluations may not accurately assess performance. This study's purpose is to evaluate differences between faculty, self-, and peer evaluations of student journal club (JC) presentations during advanced pharmacy practice experiences (APPEs).

EDUCATIONAL ACTIVITY AND SETTING

Student JC presentations for three APPE sites were identified between May 2015 and April 2018 and included if at least one faculty, self-, and peer evaluation were complete. Overall grades and individual rubric ratings were compared.

FINDINGS

Seventy-four students had complete data sets. The mean overall scores for JC presentations were 82.72%, 86.05%, and 91.01% for faculty, self-, and peer evaluations, respectively. Self-evaluation ratings were significantly higher than faculty on each domain, except for "presentation and communication skills" (mean difference: -0.2, p = .034) and "ability to answer questions" (mean difference: -0.1, p = .247). Linear regression showed a statistically significant relationship between self-evaluation and faculty evaluation ratings for ability to answer questions (beta = 0.5, p < .001). Peer evaluation ratings were significantly higher than faculty and self-evaluations (p < .05).

SUMMARY

Faculty scores on JC presentations completed during APPEs were lower compared to student evaluations of themselves and their peers. Further incorporation of self- and peer evaluation throughout pharmacy school curricula may improve student competence in performing these evaluations. Formal training is needed to improve students' ability to complete self and peer evaluations.

Changes in Pharmacy Students' Metacognition Through Self-Evaluation During Advanced Pharmacy Practice Experiences

Objective. To explore whether metacognition can be improved in Doctor of Pharmacy (PharmD) students through routine self-assessment over a year-long advanced pharmacy practice experience (APPE) sequence. Methods. Differences between self-assessment scores and preceptors' scores for three cohorts of pharmacy students between 2015 and 2018 were compared between the first, second, and third trimester to determine whether students more accurately evaluated their performance over time. The primary endpoint was change in the absolute difference between student and preceptor evaluation (rubric and composite scores) between trimesters. Results. Of 2577 student and preceptor evaluations eligible for inclusion, 1713 were completed, matched, and analyzed. Using the same rubric as preceptors, students overestimated their performance by an average of 16 points during the first trimester, followed by 14 and 12 points during the second and third trimester, respectively. This reflected a significant improvement over time. No significance difference was found between student and preceptor composite scores. Faculty preceptorship, students' pre-APPE grade point average, and type of APPE were not associated with any difference in rubric or composite scores. Conclusion. This analysis revealed that the difference between student self-evaluation grades and preceptor evaluation grades was greatest during the first trimester and significantly decreased in the second and third trimesters. This could reflect students' development of metacognitive processes over time. Metacognition is a vital skill for pharmacy students to learn, and opportunities to develop this skill should be incorporated throughout the pharmacy curricula.

Practicing metacognition through the use of repeated posttest predictions

BACKGROUND

Development of metacognitive skills is one method to increase self-awareness of pharmacy students. This study sought to assess students' posttest prediction (postdiction) of performance on a series of multiple-choice examinations to determine if feedback regarding predicted and actual performance could improve personal predictive abilities over time.

IMPACT

While there was a statistically significant change in the students' predictive abilities from examination one to examination three, lower scores in examination two disrupted the trend we had hoped to see. When broken down by overall course score, the highest performing students rarely overestimated their score (5-21% of the time, depending on examination), while the lowest performing students were more varied (22-56% over prediction, depending on examination).

RECOMMENDATIONS

This study used a novel assessment method of postdictions without additional data points such as predictions or grade point average (GPA), which could have helped confirm the value of the method. Additionally, we realized assessing the impact of the qualitative feedback students received could elucidate why and recommend this for future studies.

DISCUSSION

While students were generally poor predictors of their performance, repeated use of this skill helped them to reduce the number of over predictions made by the end of the course. This change was greatest for the lowest performing students indicating that they may receive more benefit from this exercise than higher performing students. This method of using postdictions adds to the collection of tools that can be used to measure student metacognitive skills.

Assessing students' and health professionals' competency learning from interprofessional education collaborative workshops

Interprofessional education (IPE) is a key element in preparing current and future health professionals to function in a collaborative practice-ready workforce. California State University, Fresno's College of Health and Human Services and Valley Children's Healthcare have partnered together to create an IPE collaborative that organizes and provides interagency-sponsored workshops that align learning objectives of relevant healthcare topics with 2016 IPEC core competencies for university students and health professionals. Using a pre/post design with the IPEC Competency Self-Assessment Tool, two cross-sectional studies were conducted to measure whether interprofessional learning of core competencies improved after participation in IPE workshops, and if such improvements were different between students and health professionals. In Study 1, 67 participants attended a Pediatric Head Injury IPE workshop. Of these, 19 students and 22 health professionals consented to participate and complete the IPEC survey. In Study 2, 99 participants attended an Error Disclosure IPE workshop. Of these, 26 students and 29 health professionals consented to participate and complete the IPEC survey. This investigation showed that the IPEC Competency Self-Assessment Tool discriminated competency ratings between students and health professionals in both studies and demonstrated a positive impact of IPE workshops on students' and health professionals' self-assessment of interprofessional competencies.

The Social Psychology of Biased Self-Assessment

Objective: To describe the psychological mechanisms that underlie biased self-assessment and suggest pedagogical techniques to counter them. Findings: Since the psychological mechanisms that underlie bias self-assessment occur below awareness, strategies that attempt to address bias directly are unlikely to succeed. A more effective approach may be to structure students' learning experiences in ways that prevent the unconscious biasing mechanisms from operating efficiently. Summary: Given the importance of accurate self-knowledge for professional students and clinicians, as well as its difficulty to attain, an understanding of the psychological mechanisms that contribute the most common forms of biased self-assessment is essential for creating and implementing effective mitigation strategies.

Assessing a Longitudinal Reflection Curriculum: A Case Study Of Student Self-Awareness Expressed Through Rotation Goal Achievement

The Center for the Advancement of Pharmacy Education 2013 Affective Domain mandates students develop self-awareness of personal attributes affecting professional growth. Students should self-identify needs, create and implement goals, and evaluate success. This case study describes the qualitative and quantitative findings of an intentional reflection assignment prompting students to engage in a cycle of goal writing - monitoring - reflection - new goal writing, during an immersive clinical practice experience. A blinded review of 144 student assignments is presented in the context of a curricular review of the Reflective Practitioner Program (RPP), a longitudinal reflective thread spanning four years of professional pharmacy training. Evidence gathered in the assignment review indicates that students are sufficiently capable of establishing meaningful goals and describing why the goal is important to their professional development. In contrast, students struggle with articulating strategies for goal achievement and emotions experienced during goal monitoring. In consideration of these findings, RPP faculty identified three major themes when discussing key aspects of the RPP curricular design: 1) students need to articulate strategies for goal achievement in addition to stated aims, 2) students hesitate to identify emotions when reflecting, and 3) reflection needs to be both retrospective and prospective in nature. This case study has resulted in meaningful changes to RPP curricular design and illustrates how programs may approach assessment of the Affective Domain via common curricular elements.

Comparison of faculty assessment and students' self-assessment of performance during clinical case discussions in a pharmacotherapy capstone course

OBJECTIVES

The primary objective of this study was to compare faculty assessment and third year students' self-assessment of performance in clinical case discussions. The secondary objective was to evaluate if student characteristics influence self-assessments.

METHODS

This retrospective analysis compared faculty and student self-assessment scores for two clinical case discussions using Spearman's correlation and Wilcoxon's signed ranks test. Chi-squared test was used to compare frequency of faculty and student self-assessments indicating the highest possible rating for the pooled score and for each individual component. The pooled score included three individual components: level of engagement, quality of contribution, and professionalism.

RESULTS

Pooled faculty and student self-assessments correlated for both the first (r = 0.41, p < 0.001) and second (r = 0.35; p < 0.001) clinical case discussions. The frequency that faculty and student self-assessment ratings were the highest possible pooled score was similar for both the first (51.3% vs. 44.7%, respectively, p = 0.25) and second (58.6% vs. 47.4%, p = 0.05) clinical case discussions. Student characteristics (age, gender, and grade point average at graduation) did not influence self-assessments.

CONCLUSIONS

Students' self-assessment correlated with faculty assessment of performance during clinical case discussions. Increased use of self-assessments for professional development in pharmacy and other healthcare professional curricula should be considered.

Assessing self-assessment practices: A survey of U.S. colleges and schools of pharmacy

OBJECTIVE

This study quantifies and describes student self-assessment approaches in colleges of pharmacy across the United States.

METHODS

Faculty members identified as assessment directors from college websites at U.S. colleges of pharmacy were electronically surveyed. Prior to distribution, feedback and question validation was sought from select assessment directors. Surveys were distributed and recorded, via Qualtrics^® survey software and analyzed in Microsoft Excel^®.

RESULTS

Responses were received from 49 colleges of pharmacy (n = 49/134, 37% response rate). The most commonly used strategies were reflective essays (n = 44/49, 90%), portfolios (n = 40/49, 82%), student self-evaluations (n = 35/49, 71%) and questionnaires/surveys/checklists (n = 29/49, 59%). Out of 49 submitted surveys, 35 programs noted students received feedback on self-assessment. Feedback came most commonly from faculty (n = 31/35, 88%). Thirty-four programs responded regarding self-assessment integration including fifteen colleges (n = 15/34, 44%) that integrated self-assessment both into the curriculum and co-curricular activities, while 14 (n = 14/34, 41%) integrated self-assessment exclusively into the curriculum, and five (n = 5/34, 15%) used self-assessment exclusively in co-curricular activities.

DISCUSSION AND CONCLUSIONS

Student self-assessment is a critical first step of the Continuing Professional Development (CPD) process. Colleges and schools of pharmacy use a wide variety of methods to develop this skill in preparing future practitioners.

Assessing Emotionally Intelligent Leadership in Pharmacy Students

Objective. To determine the frequency distribution of pharmacy students across Emotionally Intelligent Leadership Inventory (EILI) measures. Methods. The EILI was administered to 235 pharmacy students at two schools. The instrument was systematically compared to the 2013 CAPE Outcomes and analyzed by confirmatory factor analysis. Results. The EILI has primary connections with pharmacy competencies related to interprofessional communication and leadership. The three facets of the EILI were verified for internal consistency (Context, α=.78; Self, α=.74; Others, α=.79). Student scores were the highest for the consciousness of self facet, with a mean score of 31.4 out of 40. Conclusion. The EILI shows promise as an instrument for use in assessing pharmacy students' emotional intelligence and leadership skills.

Can Source Triangulation Be Used to Overcome Limitations of Self-Assessments? Assessing Educational Needs and Professional Competence of Pharmacists Practicing in Qatar

INTRODUCTION

Continuing professional development activities should be designed to meet the identified personal goals of the learner. This article aims to explore the self-perceived competency levels and the professional educational needs of pharmacists in Qatar and to compare these with observations of pharmacy students undergoing experiential training in pharmacies (students) and pharmacy academics, directors, and managers (managers).

METHODS

Three questionnaires were developed and administered to practicing pharmacists, undergraduate pharmacy students who have performed structured experiential training rotations in multiple pharmacy outlets in Qatar and pharmacy managers. The questionnaires used items extracted from the National Association of Pharmacy Regulatory Authorities (NAPRA) Professional competencies for Canadian pharmacists at entry to practice and measured self- and observed pharmacists' competency and satisfaction with competency level.

RESULTS

Training and educational needs were similar between the pharmacists and observers, although there was trend for pharmacists to choose more fact-intensive topics compared with observers whose preferences were toward practice areas. There was no association between the competency level of pharmacists as perceived by observers and as self-assessed by pharmacists (P ≤ .05). Pharmacists' self-assessed competency level was consistently higher than that reported by students (P ≤ .05).

DISCUSSION

The results suggest that the use of traditional triangulation might not be sufficient to articulate the professional needs and competencies of practicing pharmacists as part of a strategy to build continuing professional development programs. Pharmacists might have a limited ability to accurately self-assess, and observer assessments might be significantly different from self-assessments which present a dilemma on which assessment to consider closer to reality. The processes currently used to evaluate competence may need to be enhanced through the use of well-designed rubrics or other strategies to empower and to better inform respondents and subsequently improve their ability to self-assess their competencies.

Assessing Interprofessional Education Collaborative Competencies in Service-Learning Course

Objective. To investigate the effect of an interprofessional service-learning course on health professions students' self-assessment of Interprofessional Education Collaborative (IPEC) competencies. Design. The semester-long elective course consisted of two components: a service component where students provided patient care in an interprofessional student-run free clinic and bi-weekly workshops in which students reflected on their experiences and discussed roles, team dynamics, communication skills, and challenges with underserved patient populations. Assessment. All fifteen students enrolled in the course completed a validated 42-question survey in a retrospective post-then-pre design. The survey instrument assessed IPEC competencies in four domains: Values and Ethics, Roles and Responsibilities, Interprofessional Communication, and Teams and Teamwork. Students' self-assessment of IPEC competencies significantly improved in all four domains after completion of the course. Conclusion. Completing an interprofessional service-learning course had a positive effect on students' self-assessment of interprofessional competencies, suggesting service-learning is an effective pedagogical platform for interprofessional education.

Mapping and assessment of personal and professional development skills in a pharmacy curriculum

BACKGROUND

Health sciences programs are increasingly expanding their curricula to bridge foundational scientific knowledge with needed skills to practice and patient care. The primary objectives of this study are to 1) assess whether the personal and professional development (PPD) subdomains (self-assessment, leadership, innovation and entrepreneurship, and professionalism) are integrated in a pharmacy curriculum; and 2) identify any gaps related to the subdomains' learning objectives.

METHODS

Four different mapping activities were completed to create a comprehensive mapping plan regarding the integration of the PPD subdomains in the curriculum. The first mapping activity entailed matching the school's program educational outcomes (PEOs) to these subdomains (Step 1). Mapping of the enacted curriculum by faculty (Step 2) and learned curriculum by students (Step 3) were also completed in order to evaluate the integration of these subdomains in the curriculum. Finally, Step 4 involved mapping of the assessed curriculum by analyzing the progress of students on PPD-related competencies using standardized scoring rubrics and the correlation between students' and facultys' assessments with regard to matching competencies. The Cochrane's Q test and the Cohen's kappa coefficient were used in the statistical analysis of parametric data.

RESULTS

The subdomains were found to be woven across curricular, co-curricular, and extra-curricular activities based on the four different mapping activities. Faculty and students agreed that the PPD competencies are integrated in the curriculum; provided example courses, experiences and activities; and identified areas of further improvements. The completed mapping activities drove the development of action plans for remediation of identified gaps in the curriculum.

CONCLUSION

Mapping activities showed the sequential integration of the PPD skills at different depths and breadths in the curriculum. This study provides an example to health sciences schools on the incorporation of the PPD skills in their curricular, co-curricular and extra-curricular activities as current accreditation standards have directed Pharmacy programs to integrate and enforce them in their curricula.

Towards an Operational Definition of Clinical Competency in Pharmacy

OBJECTIVE

To estimate the inter-rater reliability and accuracy of ratings of competence in student pharmacist/patient clinical interactions as depicted in videotaped simulations and to compare expert panelist and typical preceptor ratings of those interactions.

METHODS

This study used a multifactorial experimental design to estimate inter-rater reliability and accuracy of preceptors' assessment of student performance in clinical simulations. The study protocol used nine 5-10 minute video vignettes portraying different levels of competency in student performance in simulated clinical interactions. Intra-Class Correlation (ICC) was used to calculate inter-rater reliability and Fisher exact test was used to compare differences in distribution of scores between expert and nonexpert assessments.

RESULTS

Preceptors (n=42) across 5 states assessed the simulated performances. Intra-Class Correlation estimates were higher for 3 nonrandomized video simulations compared to the 6 randomized simulations. Preceptors more readily identified high and low student performances compared to satisfactory performances. In nearly two-thirds of the rating opportunities, a higher proportion of expert panelists than preceptors rated the student performance correctly (18 of 27 scenarios).

CONCLUSION

Valid and reliable assessments are critically important because they affect student grades and formative student feedback. Study results indicate the need for pharmacy preceptor training in performance assessment. The process demonstrated in this study can be used to establish minimum preceptor benchmarks for future national training programs.

A psychometric evaluation of an advanced pharmacy practice experience clinical competency framework

OBJECTIVE

To assess the psychometric properties of the clinical competency framework known as the System of Universal Clinical Competency Evaluation in the Sunshine State (SUCCESS), including its internal consistency and content, construct, and criterion validity.

METHODS

Sub-competency items within each hypothesized competency pair were subjected to principal components factor analysis to demonstrate convergent and discriminant validity. Varimax rotation was conducted for each competency pair (eg, competency 1 vs competency 2, competency 1 vs competency 3, competency 2 vs competency 3). Internal consistency was evaluated using Cronbach alpha.

RESULTS

Of the initial 78 pairings, 44 (56%) demonstrated convergent and discriminant validity. Five pairs of competencies were unidimensional. Of the 34 pairs where at least 1 competency was multidimensional, most (91%) were from competencies 7, 11, and 12, indicating modifications were warranted in those competencies. After reconfiguring the competencies, 76 (94%) of the 81 pairs resulted in 2 factors as required. A unidimensional factor emerged when all 13 of the competencies were entered into a factor analysis. The internal consistency of all of the competencies was satisfactory.

CONCLUSION

Psychometric evaluation shows the SUCCESS framework demonstrates adequate reliability and validity for most competencies. However, it also provides guidance where improvements are needed as part of a continuous quality improvement program.

Comparison of self-reported professional competency across pharmacy education programs: a survey of Thai pharmacy graduates enrolled in the public service program

INTRODUCTION

Thai pharmacy education consists of two undergraduate programs, a 5-year Bachelor of Science in Pharmacy (BScPsci and BScPcare) degree and a 6-year Doctor of Pharmacy (Pharm D). Pharmacy students who wish to serve in the public sector need to enroll in the public service program. This study aims to compare the perception of professional competency among new pharmacy graduates from the three different pharmacy programs available in 2013 who enrolled in the public service program.

METHODS

A cross-sectional survey was conducted among new pharmacy graduates in 2013 using a self-administered, structured, close-ended questionnaire. The questionnaire consisted of respondents' characteristics and perception of professional competencies. The competency questions consisted of 13 items with a 5-point scale. Data collection was conducted during Thailand's annual health professional meeting on April 2, 2013 for workplace selection of pharmacy graduates.

RESULTS

A total of 266 new pharmacy graduates responded to the questionnaire (response rate 49.6%). There were no significant differences in sex and admission modes across the three pharmacy programs. Pharm D graduates reported highest competency in acute care services, medication reconciliation services, and primary care services among the other two programs. BScPsci graduates reported more competence in consumer health protection and herbal and alternative medicines than BScPcare graduates. There were significant differences in three competency domains: patient care, consumer protection and community health services, and drug review and information, but no significant differences in the health administration and communication domain among three pharmacy programs.

CONCLUSION

Despite a complete change into a 6-year Pharm D program in 2014, pharmacy education in Thailand should continue evolving to be responsive to the needs of the health system. An annual survey of new pharmacy graduates should be continued, to monitor changes of professional competency across different program tracks and other factors which may influence their contribution to the health service system. Likewise, a longitudinal monitoring of their competencies in the graduate cohort should be conducted.

A mentor-based portfolio program to evaluate pharmacy students' self-assessment skills

Objective. To evaluate pharmacy students' self-assessment skills with an electronic portfolio program using mentor evaluators. Design. First-year (P1) and second-year (P2) pharmacy students used online portfolios that required self-assessments of specific graded class assignments. Using a rubric, faculty and alumni mentors evaluated students' self-assessments and provided feedback. Assessment. Eighty-four P1 students, 74 P2 students, and 59 mentors participated in the portfolio program during 2010-2011. Both student groups performed well overall, with only a small number of resubmissions required. P1 students showed significant improvements across semesters for 2 of the self-assessment questions; P2 students' scores did not differ significantly. The P1 scores were significantly higher than P2 scores for 3 questions during spring 2011. Mentors and students had similar levels of agreement with the extent to which students put forth their best effort on the self-assessments. Conclusion. An electronic portfolio using mentors based inside and outside the school provided students with many opportunities to practice their self-assessment skills. This system represents a useful method of incorporating self-assessments into the curriculum that allows for feedback to be provided to the students.

Use of SMART learning objectives to introduce continuing professional development into the pharmacy curriculum

OBJECTIVE

To determine whether a 2-year continuing professional development (CPD) training program improved first-year (P1) and second-year (P2) pharmacy students' ability to write SMART (specific, measurable, achievable, relevant, and timed) learning objectives.

DESIGN

First-year students completed live or online CPD training, including creating portfolios and writing SMART objectives prior to their summer introductory pharmacy practice experience (IPPE). In year 2, P1 and P2 students were included. SMART learning objectives were graded and analyzed.

ASSESSMENT

On several objectives, the 2011 P1 students (n = 130) scored higher than did the P2 cohort (n = 105). In 2011, P2 students outscored their own performance in 2010. In 2011, P1 students who had been trained in online modules performed the same as did live-session trainees with respect to SMART objectives.

CONCLUSION

With focused online or live training, students are capable of incorporating principles of CPD by writing SMART learning objectives.

University-based continuing education for pharmacists

University-based continuing education (CE) fulfills an important role to support the professional development of pharmacists, advance the practice of pharmacy, and contribute to societal needs for research and healthcare services. Opportunities for pharmacists to engage in new models of patient care are numerous worldwide, particularly as pharmacists' scope of practice has expanded. Approaches to CE have changed to address the changing needs of pharmacists and now include a variety of approaches to support development of knowledge and skills. There is emphasis on the learning process as well as the knowledge, with the introduction of the concept of continuing professional development (CPD).As institutions of research and education, universities are uniquely positioned to bridge the gap between academic and practice environments, providing opportunities for translation of knowledge to practice. The Faculty of Pharmacy and Pharmaceutical Sciences at the University of Alberta is a provider of CE in Alberta, Canada, where an expanded scope of pharmacy practice includes prescribing, administering injections, accessing electronic patient records, and ordering laboratory tests. In this paper, the Faculty offers views about future directions for CE, including the integration of CE with core faculty activities, expanding the audience for CE, areas of focus for learning, and partnerships. Finally, we hope to ignite dialogue with others in the profession about the role and function of university-based CE.

Peer- and self-grading compared to faculty grading

OBJECTIVES

To determine the reliability and value of peer- and self -reported evaluations in the grading of pharmacy students.

METHODS

Mean student peer- and self- reported grades were compared to faculty grades in the advanced pharmacy practice experience (APPE) and seminar presentation courses. Responses from pharmacy school alumni regarding curricular peer- and self-reported evaluations were solicited using an online survey tool.

RESULTS

Self-reported student grades were lower than the faculty-reported grade overall and for the formal presentation component of the APPE course grading rubric. Self-reported grades were no different than faculty-reported grades for the seminar course. Students graded their peers higher than did faculty members for both the seminar and APPE courses on all components of the grading rubric. The majority of pharmacy alumni conducted peer- and self-evaluations (64% and 85%, respectively) at least annually and considered peer- and self-evaluations useful in assessing students' work in group projects, oral presentations, and professional skills.

CONCLUSION

The combination of self-, peer-, and faculty-assessments using a detailed grading rubric offers an opportunity to meet accreditation standards and better prepare pharmacy students for their professional careers.

Integration of pharmacy students within a level II trauma center

OBJECTIVE

To integrate fourth-year doctor of pharmacy (PharmD) students within a level II trauma center team to improve their patient care and professional communication skills.

DESIGN

PharmD students completed 2 consecutive 4-week internal medicine APPEs during the course of their fourth year, which included approximately 5 weeks working on an interprofessional trauma team. During patient rounds with the interprofessional trauma team, students provided patient care in a stepwise approach, drug information responses, patient counseling, and other services requested by team members.

ASSESSMENT

Ability-based outcomes (ABOs) assessment, faculty evaluations, and student self-assessment were conducted in the following areas: effective communication, drug therapy assessment and decision making, critical thinking and problem solving, and drug information retrieval. Students' mean score in these areas was 3.8 on a 5-point scale. Areas in which students needed improvement included: providing recommendations in a timely manner, self-confidence, identifying opportunity to verbally communicate with other team members, and addressing insecurities when answering drug information questions posed by the team.

CONCLUSION

Integrating fourth-year PharmD students within a trauma and acute surgery team and use of ABO assessment allowed for identification of areas of the curriculum in which improvements were needed, resulting in a more targeted approach earlier in the curriculum to improve students' abilities to provide appropriate and effective patient care in an interprofessional setting.