Peer- and self-grading compared to faculty grading

Evaluation of Student Peer- and Self-Grading in an Integrated Pharmacotherapy Course

OBJECTIVE

To determine if student peer- and self-grades correlate with faculty grades on case vignettes.

METHODS

This study involved first professional-year students enrolled in an Integrated Pharmacotherapy course. The course included three modified team-based learning (TBL) activities (each consisting of individual and team readiness assurance tests, followed by three open-note case vignettes completed in teams). Each student uploaded completed case vignettes to the learning management system and was assigned to complete a self- and a random, anonymous peer-grade using a provided key. Peer- and self-grades were compared to faculty grades using a null multilevel model to determine the intraclass correlation coefficient (ICC). Faculty time spent grading was captured, and students were surveyed to determine the perceived value of peer- and self-grading.

RESULTS

Faculty- and peer-grades had a slightly higher correlation than faculty- and self-grades (ICC = 0.75 vs 0.73, respectively). The ICC between all three grader groups was 0.74. Faculty spent an average of 2.5 h grading the cases after each TBL session. Students reported spending a median of 36 min on the peer- and self-grades for each TBL session. Overall, students agreed that both the self- and peer-grading activities helped identify gaps in knowledge (90% and 56%, respectively). A total of 78% of students agreed that self-grading was beneficial for their learning.

CONCLUSION

There was a moderate-to-good correlation between peer-, self-, and faculty- grades for case vignettes. Faculty time may be saved through student self- or peer-grading.

Peer review of presentations through examination software

BACKGROUND AND PURPOSE

Peer evaluations are often utilized to allow student pharmacists practice in giving and receiving feedback. In a small class setting, these can easily be completed and feedback distributed quickly. However, in the larger class setting, reviewing and disseminating peer feedback can be quite cumbersome, especially if using paper format. The purpose of this educational activity was to create a process for peer evaluations that allows for efficient collection and dissemination of peer feedback of presentations of student pharmacists and describe the student experience with this new format.

EDUCATIONAL ACTIVITY AND SETTING

In Research Topics in Pharmacy II, an electronic peer-evaluation tool was created using electronic examination software to collect and distribute this peer review in a timely fashion during and after each class session. At the completion of this course, a survey was distributed to collect student pharmacists' perception of this electronic peer-review process.

FINDINGS

A total of 63 of 91 students (69%) completed the survey. The majority of the students (98.4%) "strongly agreed" or "agreed" the peer-evaluation items made it easy to provide feedback to their peers and 79% preferred this electronic method of feedback vs. paper format. Overall, 93.6% of student pharmacists felt they were more engaged during the presentations as a result of providing electronic feedback.

SUMMARY

Maximizing our resources by creating an electronic peer evaluation with our current examination software, allowed for an efficient means of obtaining and disseminating peer review that was timely and well-received by students.

Evaluating the impact of a "grand rounds" elective course on students' perceived motivations, comfort, and confidence in providing constructive and reinforcing feedback to guest lecturers

BACKGROUND AND PURPOSE

Providing feedback is an important skill for all healthcare professionals both within and outside of their discipline. Although student pharmacists frequently receive feedback during both didactic and experiential education, training on how to provide feedback to others is less common.

EDUCATIONAL ACTIVITY AND SETTING

An elective was designed to expose second-year pharmacy students to "grand rounds" with practicing pharmacists as the presenters. Students provided feedback to presenters on presentation style and assessment questions. The primary objective of this research project was to determine if the elective improved students' motivations, comfort, and confidence in providing constructive written feedback.

FINDINGS

Over two course offerings, 54% (19 of 35) of enrolled students completed both the pre- and post-surveys. At baseline, the majority of students self-identified as being motivated, comfortable, and confident with providing quality written feedback with the exception of two specific areas: motivation to provide quality written feedback and comfort with providing difficult or sensitive written feedback. At the end of the course, the majority of students self-identified as being motivated, comfortable, and confident across all areas queried. All students agreed or strongly agreed that the efficiency and quality of their written feedback improved during the course.

SUMMARY

The course offered several benefits to students, including learning clinical topics from a variety of presenters and developing feedback skills. The implementation of the grand rounds elective provided students an opportunity to develop their motivation, comfort, and confidence with providing quality constructive written feedback.

Pharmacy students' intrinsic motivation to participate in electronic peer review of journal article critique assignments

BACKGROUND AND PURPOSE

Journal article critiques are frequently used to teach literature evaluation to pharmacy students. Peer review is one method to improve students' competency regarding journal article critiques. The objective of this manuscript is to describe implementation of electronic peer review of journal article critique drafts and explore students' intrinsic motivation to participate in the peer-review process. Influence of students' motivation to participate in peer review on their self-competence regarding journal article critiques was also explored.

EDUCATIONAL ACTIVITY AND SETTING

Second-year pharmacy students participated in three in-class, electronic, anonymous peer-review sessions for written journal article critique drafts. Students were invited to complete a 16-item survey instrument based on self-determination theory. Modified Intrinsic Motivation Inventory and Perceived Competence Scales assessed student interest in and perceived value of the peer-review sessions and their self-competence regarding journal article critiques.

FINDINGS

The survey response rate was 99% (146/148). Based on a Likert scale of 1 (not at all true) to 7 (very true), students demonstrated moderate interest in the peer-review sessions (M = 3.86), viewed them as valuable (M = 5.25), and had a moderate level of self-competence regarding journal article critiques (M = 3.74). Additionally, interest and perceived value of the peer-review sessions significantly influenced self-competence in completing journal article critiques.

SUMMARY

An anonymous, electronic peer-review system provides an efficient method for in-class peer review of draft assignments. Implementing strategies to increase student interest in peer review may increase their motivation for participation and ultimately improve self-competence regarding literature evaluation.

A prediction-based method to estimate student learning outcome: Impact of response rate and gender differences on evaluation results

BACKGROUND

Low response rates threaten the reliability and validity of student evaluations of teaching. Previous research has shown that asking students to predict how satisfied their fellow students were with a course produces reliable results at lower response rates. The aim of this study was to investigate whether this prediction-based method can also be used to evaluate student learning outcome.

METHODS

Before and after a cardiorespiratory module, 128 fourth-year medical students provided self-assessments and predictions of performance on 27 specific learning objectives and took formative tests on the respective contents. Pre-post performance gain was compared across all three modalities.

RESULTS

Formative exam results indicated a performance gain of 63.0%. Self-assessed and prediction-based performance gains were identical (67.8%) but both slightly overestimated actual performance gain. Irrespective of the method used, a response rate of 20% was sufficient to produce reliable results. Compared to male students, females greatly overestimated their peers' performance which led to inflated performance gain values.

CONCLUSIONS

Student self-assessments and predictions are equally valid sources of learning outcome measures, and low response rates are sufficient to produce stable results. When using a prediction-based approach, a tendency to overestimate learning outcome in female students needs to be taken into account.

Peer Assessment in Medical Student Education: A Study of Feasibility, Benefit, and Worth

BACKGROUND

Direct experience with medical procedures is an important component of medical school training, yet opportunities for medical students have dwindled for various reasons. To offset this, simulated procedures are being integrated into training. However, this comes with additional time commitments required of teaching surgeons regarding assessment of simulation. A solution to this could be peer assessment. We hypothesize that there will be no significant difference between peer assessment when compared to that of a teaching surgeon.

METHODS

Third-year medical students were shown 3 simulated procedures by teaching surgeon and provided a grading rubric. Student performances were independently graded by peer assessment and by teaching surgeons. All peer assessment grades and surgeon grades were compared.

RESULTS

Four hundred fifty-nine medical students completed the simulation procedures. Comparisons between the teaching surgeons and peer assessment evaluations demonstrated a 99% interobserver agreement for pass-fail designation and 98% agreement for individual data points (kappa = .78). Survey results demonstrated a significant increase in confidence in performing the tested items and comfort with peer assessment.

DISCUSSION

This analysis demonstrates that the inclusion of peer assessment within medical school is highly comparable to teaching surgeon assessments.

Student Potential for Self-assessment in a Clinical Dentistry Practical Training Course on Communication Skills

This study aimed to investigate student potential for self-assessment in a clinical dentistry practical training course focused on communication skills. Participants were 124 fourth-year students (70 males, 54 females; all Japanese) in 2017 and 2018 at the Nippon Dental University, School of Life Dentistry at Niigata. Participating students belonged to different cohorts in 2017 and 2018. Participants were asked to complete a self-evaluation sheet at the end of each unit of the course. Their self-evaluation scores and the faculty evaluation scores for each student for Units 1-1, 1-2, and 1-3 were statistically analyzed. The results showed that females tended to rate themselves significantly higher than males. Furthermore, there were significant differences in evaluation scores between students and faculty for nine of 11 evaluation items for male students and 10 of 11 items for female students in Unit 1-3. Faculty expectations increased from Unit 1-1 to Unit 1-3, although students were satisfied with their performance and had a sense of achievement. However, students' actual performance was below faculty expectations, suggesting faculty evaluations were stricter than students' self-evaluation. Self-assessment may enhance students' ability for self-directed learning and may also inform how faculty can effectively educate dental students. Dental educators should support students to increase their levels of self-efficacy, which will enhance their self-evaluation skills.

Physiotherapy students' self-assessment of performance-Are there gender differences in self-assessment accuracy?

BACKGROUND AND PURPOSE

The ability to critically appraise one's performance is paramount in physiotherapy and, although there is a paucity of research in this area, factors such as gender have been suggested to moderate the self-assessment accuracy of healthcare students. The purpose of this study was to determine a posteriori self-assessment accuracy of first-year physiotherapy students following a multiple-choice anatomy examination and to determine the specific influence of gender as a potential moderator of self-assessment accuracy.

METHOD

One-hundred-and-forty-two students (n = 72 female) enrolled in their second semester of a three-and-a-half-year physiotherapy programme participated in the study. A purpose-made self-assessment questionnaire was used to measure the students' self-assessment ability, estimating their performance on 11 different anatomical categories following the examination. This was then compared with a criterion measure matched with the questionnaire. The accuracy of the students' self-assessment was investigated by the relation between self-assessment and objective performance.

RESULTS

The study showed low-to-moderate self-assessment accuracy (rho ranging from 0.318 to 0.675) with the students underestimating their performance in six out of 11 categories (p < 0.019). Gender did not contribute significantly to differences in accuracy between students' self-assessment and the criterion measure (p = 0.474).

CONCLUSION

According to the results, the students demonstrated low-to-moderate self-assessment accuracy when compared to their performance. Notably, the results clearly showed that gender did not function as a moderator of self-assessment accuracy among first-year physiotherapy students.

Assessment of student performance on a mock new prescription counseling session and an objective structured clinical examination across five years

INTRODUCTION

New prescription counseling (NPC) provides vital information to patients regarding newly prescribed medications to improve treatment outcomes. This evaluation's goals were to describe the implementation of teaching and assessment methods incorporated into an NPC module across two courses at Auburn University and evaluate student performance on assessments of NPC skills over five years.

METHODS

Assessments included self and peer evaluations of a recorded mock NPC session, NPC objective structured clinical examination (OSCE), and overall course grade. Scores were analyzed over a five-year period (2012 to 2016) to evaluate mean student performance in each of four domains (gathering information, communication, management strategies, and monitoring and follow-up) and overall for each assessment. Pearson's correlation coefficients between these scores were calculated by combining all five years of data.

RESULTS

Seven-hundred thirty-three students were included in the analysis. No trends in mean domain and total assessment scores were noted across years. Self and peer evaluation domain and total scores were significantly, though moderately, correlated across assessments (r = 0.43-0.51), except in the communication domain (r = 0.12). Overall, NPC OSCE total scores were not correlated with NPC recording self or peer evaluation total scores (r = 0.06 and r = 0.11, respectively). Assessment total scores were all moderately correlated with overall course grades.

CONCLUSIONS

Utilization of a scaffolding approach with class discussion, demonstration, role-play, self and peer evaluation, and OSCEs, is an effective means of building student competence in patient counseling for the NPC context.

Grit in Medical Education: Differing Perspectives of Residents and Mentors

Background Increasing concerns about depression and burnout in residents have led to a recent focus on assessing "non-cognitive" traits in residents and residency applicants. One attribute that has received significant attention is grit, defined as trait-level perseverance and passion for long-term goals^. With an objective measure available, an important question is under what circumstances of administration is that measure reliable and accurate. The goal of this study was to ascertain whether internal medicine residents and their faculty mentors were congruent in their ratings of resident grit, or if not, how the ratings differed.  Methods Subjects were internal medicine residents (N=42) at a community-based university-affiliated hospital internal medicine residency program. Near the end of the academic year 2019, residents completed the GRIT-S (short form). As each resident is assigned a mentor during their training, each resident's mentor was also asked to complete the GRIT-S based on their view of their mentee.  Results This study failed to find a significant correlation between resident self-ratings of grit and those of their mentors. Conclusions The results of these two studies underscore the difficulty in obtaining accurate assessments of non-cognitive traits. These results further the understanding of the role of grit and raise important questions about how assessments might be used to assure validity. Further areas of inquiry into this potentially important characteristic are suggested.

Comparing faculty evaluations of student journal club presentations with student self- and peer evaluations during advanced pharmacy practice experiences

BACKGROUND AND PURPOSE

Self-evaluation is a critical step in professional development. Peer evaluation may enhance student learning and help peer evaluators recognize their own limitations. However, these evaluations may not accurately assess performance. This study's purpose is to evaluate differences between faculty, self-, and peer evaluations of student journal club (JC) presentations during advanced pharmacy practice experiences (APPEs).

EDUCATIONAL ACTIVITY AND SETTING

Student JC presentations for three APPE sites were identified between May 2015 and April 2018 and included if at least one faculty, self-, and peer evaluation were complete. Overall grades and individual rubric ratings were compared.

FINDINGS

Seventy-four students had complete data sets. The mean overall scores for JC presentations were 82.72%, 86.05%, and 91.01% for faculty, self-, and peer evaluations, respectively. Self-evaluation ratings were significantly higher than faculty on each domain, except for "presentation and communication skills" (mean difference: -0.2, p = .034) and "ability to answer questions" (mean difference: -0.1, p = .247). Linear regression showed a statistically significant relationship between self-evaluation and faculty evaluation ratings for ability to answer questions (beta = 0.5, p < .001). Peer evaluation ratings were significantly higher than faculty and self-evaluations (p < .05).

SUMMARY

Faculty scores on JC presentations completed during APPEs were lower compared to student evaluations of themselves and their peers. Further incorporation of self- and peer evaluation throughout pharmacy school curricula may improve student competence in performing these evaluations. Formal training is needed to improve students' ability to complete self and peer evaluations.

Changes in Pharmacy Students' Metacognition Through Self-Evaluation During Advanced Pharmacy Practice Experiences

Objective. To explore whether metacognition can be improved in Doctor of Pharmacy (PharmD) students through routine self-assessment over a year-long advanced pharmacy practice experience (APPE) sequence. Methods. Differences between self-assessment scores and preceptors' scores for three cohorts of pharmacy students between 2015 and 2018 were compared between the first, second, and third trimester to determine whether students more accurately evaluated their performance over time. The primary endpoint was change in the absolute difference between student and preceptor evaluation (rubric and composite scores) between trimesters. Results. Of 2577 student and preceptor evaluations eligible for inclusion, 1713 were completed, matched, and analyzed. Using the same rubric as preceptors, students overestimated their performance by an average of 16 points during the first trimester, followed by 14 and 12 points during the second and third trimester, respectively. This reflected a significant improvement over time. No significance difference was found between student and preceptor composite scores. Faculty preceptorship, students' pre-APPE grade point average, and type of APPE were not associated with any difference in rubric or composite scores. Conclusion. This analysis revealed that the difference between student self-evaluation grades and preceptor evaluation grades was greatest during the first trimester and significantly decreased in the second and third trimesters. This could reflect students' development of metacognitive processes over time. Metacognition is a vital skill for pharmacy students to learn, and opportunities to develop this skill should be incorporated throughout the pharmacy curricula.

A systematic approach to team creation and peer evaluation in a large classroom setting

INTRODUCTION

There is limited data to support a particular method for optimal team creation in pharmacy education. We aimed to implement and evaluate a systematic approach to team creation and compare the impact on team dynamics to teams created via random selection.

METHODS

Two concurrent courses were used to assess team creation methods. Student-specific variables were used for team creation in one course while another course utilized teams created via random allocation. Each course conducted similar peer evaluations to provide feedback and assess team dynamics. A matched pre- and post-course survey assessed changes in student perceptions of team creation and peer evaluation. Student-perceived team effectiveness and individual coursework performance were assessed.

RESULTS

A total of 109 students were enrolled in each of the two courses, with 98% and 93% providing evaluation of team dynamics within the intervention and control methods, respectively. Students perceived better team effectiveness in intervention-created groups in relation to decreased task (p < 0.01), relationship (p < 0.01), and process conflict (p < 0.01), along with increased task attraction (p < 0.01). There was no significant difference in student performance on individual examinations team creation methods (p = 0.17).

CONCLUSIONS

A systematic approach to team creation improved student-perceived team dynamics with no significant impact on coursework scores. A systematic approach to team creation via a web-based platform is feasible in a large classroom setting and may provide an avenue for assessment approaches related to teamwork and team dynamics.

Evaluation of a peer- and self-grading process for clinical writing assignments

OBJECTIVES

As class sizes in pharmacy education increase, faculty must develop new assessment strategies for essay writing assignments. This study evaluated accuracy and student perceptions of an innovative grading process that utilizes both peer- and self-assessment.

METHODS

Four SOAP note sessions were evaluated. Each session included four activities: a writing workshop, assessment workshop, and reflection session. For each assessment workshop students scored their note and a blinded peer's note using a grading form, facilitated by a faculty-led discussion. In a subsequent reflection session, students reviewed their peer- and self-assigned grades and could petition for faculty review if desired. The average self-, peer-, and final-grades were compared for each of the four SOAP note sessions using ANOVA. After the fourth session, students completed an anonymous 10-question Likert-scale survey regarding their perceptions of the process and three open-response questions. Survey results were analyzed with descriptive statistics.

RESULTS

Approximately 140 students participated. No difference was found between the average self-, peer-, and final-grades for all four sessions (p > 0.05). The survey response rate was 65% (91/140). Survey questions were grouped into three themes. The majority of students either strongly agreed or agreed that sessions were well organized and effective (≥84%), assessment workshops enhanced learning (≥68%), and the scoring method was fair (≥72%). The lowest score (mean 2.53 on a 4-point scale) reflected satisfaction with peer-provided feedback. When asked what they liked most, respondents most commonly cited that faculty-led review and discussion enhanced clinical knowledge.

IMPLICATIONS

The combination of a peer- and self-assessment process was accurate, well-received, and can be used to decrease faculty workload.

Implementation of Hybrid Learning in Biosignal Measurement and Instrumentation Course to Improve Student Outcomes

The mixed-mode approach is introduced in the Biosignal Measurement and Instrumentation course to achieve student outcome. This learning scheme combines the traditional learning with project-based learning (PBL) method. The purpose of this study is to compare the self/group-, peer-, and instructor assessment in PBL using ANOVA. There are 16 groups of two students that are participated in this study. The study provides further evidence that there is significant difference score between student-based assessment (group- and peer-assessment) and instructor assessment. The explanation for the result is this the first time for the student to do student-based assessment using a rubric. The students still confuse how to give the scores, even though there is a detailed rubric. From 10 project criteria, there are only three, project background; project demonstration; and answer, that have p- value greater than significant level (α = 0.01). Other criteria, review; specifications; block diagram; detailed design and implementation; conclusion or discussion; citations; and clarity of writing, have a p-value less than 0.01. This paper has highlighted the importance of the instructor role to monitor the project progress properly during mentoring sessions.

Reliability of peer assessment of patient education simulations

BACKGROUND

At Belmont University College of Pharmacy, the final introductory pharmacy practice experience (IPPE) course in the IPPE series, IPPE V, is designed to assess readiness for advanced pharmacy practice experiences and includes three patient counseling simulations. These simulations have required greater resources. The objective of our study was to determine if student performance on patient counseling simulations can be accurately assessed by peers.

EDUCATIONAL ACTIVITY

Students were required to participate in patient counseling simulations throughout the semester. For each simulation, students were assigned one role: pharmacist, patient, or peer-evaluator. Each pharmacist counseled the patient on a specific product while the peer-evaluator assessed the accuracy and completeness of the counseling using a detailed checklist. The patient used a checklist to assess the pharmacist's communication skills. Faculty assessed the student evaluators and the patients by counting the number of discrepancies between the student evaluator's and the live faculty checklists. Students were surveyed at the end of the semester regarding their beliefs and perceptions of peer assessment for the communication simulations.

CRITICAL ANALYSIS OF THE EDUCATIONAL ACTIVITY

Of 65 students enrolled in the spring 2018 course, complete recordings and checklists were available for 54 simulations (83.1%). Interrater reliability was high with all correlation coefficients exceeding 0.86. Students agreed that they were comfortable assessing patient education content (82.14%) and communication skills (82.14%). Our results indicate that peer evaluation during patient education simulation is reliable and acceptable to students.

Comparing student self-assessments of global communication with trained faculty and standardized patient assessments

BACKGROUND AND PURPOSE

Assess the reliability of first year pharmacy student assessments completed by faculty members in comparison with a standardized patient (SP), and student self-assessments during a structured educational module on communication.

EDUCATIONAL ACTIVITY AND SETTING

Pharmacy students completed four stations focused on communication with an SP. During each encounter, students completed a self-assessment and were evaluated by a faculty member and a trained SP. A five point Likert scale was used to evaluate student performance. Faculty assessments were compared against all others. A Pearson correlation coefficient for total scores was used and a Cohen's kappa was used to compare inter-rater reliability. Agreement and correlation was performed with student results categorized into poor, adequate, and exceptional performance based on faculty evaluation.

FINDINGS

Twenty-four students participated. In all stations, student self-assessments were graded higher than corresponding faculty and SP assessments. Agreement between faculty, SP, and self-assessment was fair to slight (k < 0.4) for all comparisons but only significant (p < 0.05) between the faculty and self-assessment. After categorization, there was a small, non-significant correlation between faculty and self-assessment (r = 0.13, p = 0.21) and moderate and significant correlation between faculty and SP (r = 0.32, p = 0.001). Categorized inter-rater agreement was fair for all comparisons (k < 0.2) and only significant (p < 0.05) between faculty and SP assessment.

DISCUSSION

Pharmacy students in their first professional year assess their communication skills more positively than other evaluators. Further instruction for students and reflection may be required to build understanding of global assessment in communication.

SUMMARY

There is high incongruity between student self-assessment and faculty appraisal.

Comparison of faculty assessment and students' self-assessment of performance during clinical case discussions in a pharmacotherapy capstone course

OBJECTIVES

The primary objective of this study was to compare faculty assessment and third year students' self-assessment of performance in clinical case discussions. The secondary objective was to evaluate if student characteristics influence self-assessments.

METHODS

This retrospective analysis compared faculty and student self-assessment scores for two clinical case discussions using Spearman's correlation and Wilcoxon's signed ranks test. Chi-squared test was used to compare frequency of faculty and student self-assessments indicating the highest possible rating for the pooled score and for each individual component. The pooled score included three individual components: level of engagement, quality of contribution, and professionalism.

RESULTS

Pooled faculty and student self-assessments correlated for both the first (r = 0.41, p < 0.001) and second (r = 0.35; p < 0.001) clinical case discussions. The frequency that faculty and student self-assessment ratings were the highest possible pooled score was similar for both the first (51.3% vs. 44.7%, respectively, p = 0.25) and second (58.6% vs. 47.4%, p = 0.05) clinical case discussions. Student characteristics (age, gender, and grade point average at graduation) did not influence self-assessments.

CONCLUSIONS

Students' self-assessment correlated with faculty assessment of performance during clinical case discussions. Increased use of self-assessments for professional development in pharmacy and other healthcare professional curricula should be considered.

Assessing self-assessment practices: A survey of U.S. colleges and schools of pharmacy

OBJECTIVE

This study quantifies and describes student self-assessment approaches in colleges of pharmacy across the United States.

METHODS

Faculty members identified as assessment directors from college websites at U.S. colleges of pharmacy were electronically surveyed. Prior to distribution, feedback and question validation was sought from select assessment directors. Surveys were distributed and recorded, via Qualtrics^® survey software and analyzed in Microsoft Excel^®.

RESULTS

Responses were received from 49 colleges of pharmacy (n = 49/134, 37% response rate). The most commonly used strategies were reflective essays (n = 44/49, 90%), portfolios (n = 40/49, 82%), student self-evaluations (n = 35/49, 71%) and questionnaires/surveys/checklists (n = 29/49, 59%). Out of 49 submitted surveys, 35 programs noted students received feedback on self-assessment. Feedback came most commonly from faculty (n = 31/35, 88%). Thirty-four programs responded regarding self-assessment integration including fifteen colleges (n = 15/34, 44%) that integrated self-assessment both into the curriculum and co-curricular activities, while 14 (n = 14/34, 41%) integrated self-assessment exclusively into the curriculum, and five (n = 5/34, 15%) used self-assessment exclusively in co-curricular activities.

DISCUSSION AND CONCLUSIONS

Student self-assessment is a critical first step of the Continuing Professional Development (CPD) process. Colleges and schools of pharmacy use a wide variety of methods to develop this skill in preparing future practitioners.

Teaching students to become effective educators through an academic elective course

INTRODUCTION

2013 CAPE outcomes and 2016 ACPE accreditation standards state student graduates should be "able to educate all audiences by determining the most effective and enduring ways to impart information and assess learning".

RATIONALE AND OBJECTIVE

The purpose of this article is to describe the implementation of a professional elective that achieves this outcome and improves students' abilities to teach, communicate, and conduct peer and self-assessment. The Learn to Teach course is a 2-credit hour elective offered to second professional year pharmacy students over an 11-week quarter.

METHODS

Quantitative and qualitative data was collected from 2010 to 2015 and included students' overall course grades, grades on the formal presentation, and student course evaluations. Peer and self-assessment scores were compared to faculty scores to see if there was any correlation.

RESULTS

116 students have participated in the elective course from 2010 to 2015, 64% of which completed a course evaluation. The mean grade achieved in the course was 93.5%. The mean faculty formal presentation grade for the course was 90.9% and the mean peer assessment grade was 96.2%. The peer and faculty presentation grades showed a 23.8% correlation. Overall, students strongly agreed/agreed that the course objectives were clear (average score of 3.79/4) and course assessments correlated with the course objectives (average score of 3.81/4).

CONCLUSION

The academic elective has been well received and successfully completed by professional students over the last several years. The skills taught in this academic elective course are applicable to a variety of future pharmacy careers.

Can Source Triangulation Be Used to Overcome Limitations of Self-Assessments? Assessing Educational Needs and Professional Competence of Pharmacists Practicing in Qatar

INTRODUCTION

Continuing professional development activities should be designed to meet the identified personal goals of the learner. This article aims to explore the self-perceived competency levels and the professional educational needs of pharmacists in Qatar and to compare these with observations of pharmacy students undergoing experiential training in pharmacies (students) and pharmacy academics, directors, and managers (managers).

METHODS

Three questionnaires were developed and administered to practicing pharmacists, undergraduate pharmacy students who have performed structured experiential training rotations in multiple pharmacy outlets in Qatar and pharmacy managers. The questionnaires used items extracted from the National Association of Pharmacy Regulatory Authorities (NAPRA) Professional competencies for Canadian pharmacists at entry to practice and measured self- and observed pharmacists' competency and satisfaction with competency level.

RESULTS

Training and educational needs were similar between the pharmacists and observers, although there was trend for pharmacists to choose more fact-intensive topics compared with observers whose preferences were toward practice areas. There was no association between the competency level of pharmacists as perceived by observers and as self-assessed by pharmacists (P ≤ .05). Pharmacists' self-assessed competency level was consistently higher than that reported by students (P ≤ .05).

DISCUSSION

The results suggest that the use of traditional triangulation might not be sufficient to articulate the professional needs and competencies of practicing pharmacists as part of a strategy to build continuing professional development programs. Pharmacists might have a limited ability to accurately self-assess, and observer assessments might be significantly different from self-assessments which present a dilemma on which assessment to consider closer to reality. The processes currently used to evaluate competence may need to be enhanced through the use of well-designed rubrics or other strategies to empower and to better inform respondents and subsequently improve their ability to self-assess their competencies.

Assessment of a Revised Method for Evaluating Peer-graded Assignments in a Skills-based Course Sequence

OBJECTIVE. To evaluate the modified peer-grading process incorporated into the SOAP (subjective, objective, assessment, plan) note sessions in a skills-based pharmacy course sequence. DESIGN. Students assessed a de-identified peer's SOAP note in a faculty-led peer-grading session followed by an optional grade challenge opportunity. Using paired t tests, final session grades (peer-graded with challenge opportunity) were compared with the retrospective faculty-assigned grades. Additionally, students responded to a survey using 4-point Likert scale and open-answer items to assess their perceptions of the process. ASSESSMENT. No significant difference was found between mean scores assigned by faculty members vs those made by student peers after participation in 3 SOAP note sessions, which included a SOAP note-writing workshop, a peer-grading workshop, and a grade challenge opportunity. The survey data indicated that students generally were satisfied with the process. CONCLUSION. This study provides insight into the peer-grading process used to evaluate SOAP notes. The findings support the continued use of this assessment format in a skills-based course.

Medical Students' Perceptions of Peer Assessment in a Problem-based Learning Curriculum

OBJECTIVES

Peer assessment (PA) is believed to support learning and help students develop both professionally and personally. The aim of this study was to examine medical students' perceptions of intragroup PA in a problem-based learning (PBL) setting.

METHODS

This study was carried out between September and November 2014 and involved six random groups of fourth-year undergraduate medical students (n = 60) enrolled at the Arabian Gulf University in Manama, Bahrain. While working on set tasks within a curriculum unit, each student evaluated a randomly selected peer using an English language adapted assessment tool to measure responsibility and respect, information processing, critical analysis, interaction and collaborative skills. At the end of the unit, students' perceptions of PA were identified using a specifically-designed voluntary and anonymous self-administered questionnaire in English.

RESULTS

A total of 55 students participated in the study (response rate: 92%). The majority of students reported that their learning (60%), attendance (67%), respect towards group members (70%) and participation in group discussions (71%) improved as a result of PA. Regarding problem analysis skills, most participants believed that PA improved their ability to analyse problems (65%), identify learning needs (64%), fulfil tasks related to the analysis of learning needs (72%) and share knowledge within their group (74%). Lastly, a large proportion of students reported that this form of assessment helped them develop their communication (71%) and self-assessment skills (73%), as well as collaborative abilities (75%).

CONCLUSION

PA was well accepted by the students in this cohort and led to self-reported improvements in learning, skills, attitudes, engagement and other indicators of personal and professional development. PA was also perceived to have a positive impact on intragroup attitudes.

Using the Jigsaw Technique to Teach Clinical Controversy in a Clinical Skills Course

OBJECTIVE

To evaluate the effectiveness and student perception of the jigsaw technique to engage students in a clinical controversy exercise and to assess student engagement level during each step of the process.

DESIGN

Students were assigned individual readings pertaining to the controversy surrounding the drug oxybutynin switching from prescription to nonprescription. They met with an expert group and teaching groups during mandatory laboratory time and worked together to formulate a recommendation on the appropriateness of nonprescription conversion for a drug.

ASSESSMENT

A quiz taken individually was used to measure effectiveness. Student perception and level of engagement was assessed using surveys.

CONCLUSION

The jigsaw technique was successful in teaching the concepts involved in the clinical controversy. Group members rated themselves and fellow participants' level of engagement as high during both the expert group and teaching group sessions. Most students reported they learned about the same or more with the jigsaw technique compared to another cooperative learning technique used in the curriculum.

A workshop series using peer-grading to build drug information, writing, critical-thinking, and constructive feedback skills

OBJECTIVE

To utilize a skills-based workshop series to develop pharmacy students' drug information, writing, critical-thinking, and evaluation skills during the final didactic year of training.

DESIGN

A workshop series was implemented to focus on written (researched) responses to drug information questions. These workshops used blinded peer-grading to facilitate timely feedback and strengthen assessment skills. Each workshop was aligned to the didactic coursework content to complement and extend learning, while bridging and advancing research, writing, and critical thinking skills.

ASSESSMENT

Attainment of knowledge and skills was assessed by rubric-facilitated peer grades, faculty member grading, peer critique, and faculty member-guided discussion of drug information responses. Annual instructor and course evaluations consistently revealed favorable student feedback regarding workshop value.

CONCLUSION

A drug information workshop series using peer-grading as the primary assessment tool was successfully implemented and was well received by pharmacy students.