Beyond Collaborative Learning: a Comparison of Small Groups in Face-to-Face and Online Settings

Small group-based instructional approaches such as case-based learning (CBL) and team-based learning (TBL) are widely used in medical education to promote collaboration and team learning. During the pandemic, many medical schools shifted from face-to-face instruction to online settings. While CBL/TBL are intended to foster collaborative skills, it is unclear how its use evolves in an online setting and whether the online setting impacts students' perceptions and behaviors in collaboration. This study examined how the change from in-person to online CBL/TBL impacted students' collaboration. We used a mixed-methods sequential design, first collecting and analyzing retrospective cohort quantitative data with the Class of 2023 through peer evaluation surveys followed by six focus groups. Students' assessment of their peers was generally positive. Nonparametric testing showed significant differences for two questions with less positive perceptions in the virtual setting compared to when students had in-person CBL/TBL. The focus group results identified several themes related to collaboration and learning communities. In the virtual setting, students not only lost collaboration opportunities with their group members in CBL/TBL, but also learning opportunities and social connections with other groups and the community as a whole. Virtual learning environments may have presented challenges for collaborative learning and the establishment of a sense of community.

Peer evaluation of teaching programs within pharmacy education: A review of the literature

BACKGROUND

Student evaluations of teaching are one method to evaluate faculty teaching, but alone may have bias. Peer evaluation of teaching represents an important approach in assessing and providing feedback to faculty, however, numerous strategies exist for implementation. This review was conducted to identify existing peer evaluation of teaching programs and to describe their development, implementation, and outcomes.

METHODS

A literature search was conducted through January 2023 using the following search terms alone or in combination: peer evaluation, peer evaluation of teaching, peer review of teaching, pharmacy, pharmacy education, teaching evaluation, peer assessment. Articles were included that described and evaluated the development and/or specific outcomes of individual peer evaluation of teaching programs in the didactic setting within pharmacy education.

RESULTS

Eight articles were identified for review; six described peer evaluation of teaching in individual programs and two described components of the same program. Classroom observation was the core component of all programs and some but not all programs included pre- and/or post-observation meetings. Outcomes of programs included perception of participants via survey in seven studies with generally positive perceptions noted. Time burden was the most common concern noted for the implementation of the peer evaluation programs.

IMPLICATIONS

Limited literature exists describing peer evaluation of teaching programs and outcomes within in pharmacy education. Peer evaluation processes are individualized and vary across institutions. Opportunities exist to develop time-efficient programs to further assess the outcomes of peer evaluation of teaching and compare with student evaluations or other tools to evaluate education quality.

Assessment of the validity of peer scores and peer feedback in an online peer assessment platform (Kritik)

BACKGROUND AND PURPOSE

Online peer assessment platforms aim to improve the accuracy of peer scores (numerical scores based on a rubric) and hold students accountable for peer feedback (written comments). We evaluated the validity of peer scores and peer feedback using the online platform, Kritik.

EDUCATIONAL ACTIVITY AND SETTING

The course was a two-credit hour online infectious diseases pharmacotherapy elective consisting of twelve third-year students enrolled in a four-year doctor of pharmacy program. Students analyzed patient cases and created video presentations of their therapeutic care plan weekly. Each student scored three peers' presentations using a rubric and provided peer feedback in Kritik. The instructor independently scored the presentations. The students' presentation scores, which was the weighted average of three peers' scores, was compared to the instructor's score. Students also rated the peer feedback they received using two Likert-type scales (feedback-on-feedback [FoF] ratings). Two faculty rated 97 randomly selected peer feedback (written comments) and recorded their FoF ratings separately. Students completed an anonymous course evaluation and exit survey.

FINDINGS

The Pearson correlation coefficient between weighted peer scores and instructor scores for 91 presentations was r = 0.880. There was fair agreement in FoF ratings between students and faculty based on weighted kappa. All students would recommend the course and had a positive experience with peer assessment and the platform.

SUMMARY

Weighted peer scores strongly correlated with instructor scores, and students held each other accountable for peer feedback in Kritik. Our findings should be confirmed in different contexts and settings.

Effect of prior knowledge and peer evaluation ratings on final exam performance in a team-based learning chemistry course

INTRODUCTION

Various reports have been published regarding adoption of team-based learning (TBL) in pharmacy education. However, there is insufficient published evidence on the effect of student characteristics on student learning outcomes in a TBL curriculum. The purpose of this research was to evaluate the effects of pre-study examination results and peer evaluation ratings on learning outcomes.

METHODS

The TBL strategy was adopted for a basic chemistry exercise for first-year students at a private pharmacy school in Japan (2012-2019). For the analysis, students were divided into four analytical groups according to quartiles of pre-study examination results. The students were further divided into the high-peer evaluation rating group and low-peer evaluation rating group. We compared the final exam performance results between these groups.

RESULTS

In all four groups by quartiles of pre-study examination results, the course final exam performance was higher for the high-peer evaluation rating students than for the low-peer evaluation rating students.

CONCLUSIONS

Within the TBL framework, students with higher peer evaluation scores performed better on the final exam, regardless of the pre-study examination results.

Peer evaluations of group work in different years of medical school and academic achievement: how are they related?

BACKGROUND

To develop the skills needed in health care teams, training communication and teamwork skills are important in medical education. Small group collaborative learning is one of the methods utilized in such trainings, and peer evaluation is suggested to be useful in reinforcing the effectiveness of group learning activities. In Mie University Faculty of Medicine, group work consisting of book review sessions of liberal arts education in the first grade and problem-based learning (PBL) sessions in preclinical years were conducted using the same peer evaluation system that included three domains: degree of prior learning, contribution to group discussion, and cooperative attitude. This study was conducted to determine the relationships among behaviors during group work and the academic achievement of medical students.

METHODS

With the data from a cohort of medical students in three consecutive academic years (n = 340), peer evaluation scores in groupworks of book review sessions, those in PBL sessions and paper test scores of preclinical years were analyzed. The correlations were analyzed with Spearman's correlation coefficient, and the respective scores were compared by using the Wilcoxon signed-ranked test.

RESULTS

Significant correlations were observed among the evaluation scores of respective domains in group work and paper test scores. The degree of prior learning had the strongest relationship among the three domains (r_s = 0.355, p < 0.001 between book review sessions and PBL; r_s = 0.338, p < 0.001 between book review sessions and paper test score; r_s = 0.551, p < 0.001 between PBL and paper test score). Peer evaluation scores of respective domains were found to be significantly higher in PBL.

CONCLUSION

Medical students maintained their groupwork behaviors to some extent from early school to preclinical years. Those behaviors were positively related to their academic achievement in the later years of the medical education curriculum. Our study highlighted the importance of the early introduction of group work. The results will be useful to motivate medical students to put more effort into group work.

Peer review of presentations through examination software

BACKGROUND AND PURPOSE

Peer evaluations are often utilized to allow student pharmacists practice in giving and receiving feedback. In a small class setting, these can easily be completed and feedback distributed quickly. However, in the larger class setting, reviewing and disseminating peer feedback can be quite cumbersome, especially if using paper format. The purpose of this educational activity was to create a process for peer evaluations that allows for efficient collection and dissemination of peer feedback of presentations of student pharmacists and describe the student experience with this new format.

EDUCATIONAL ACTIVITY AND SETTING

In Research Topics in Pharmacy II, an electronic peer-evaluation tool was created using electronic examination software to collect and distribute this peer review in a timely fashion during and after each class session. At the completion of this course, a survey was distributed to collect student pharmacists' perception of this electronic peer-review process.

FINDINGS

A total of 63 of 91 students (69%) completed the survey. The majority of the students (98.4%) "strongly agreed" or "agreed" the peer-evaluation items made it easy to provide feedback to their peers and 79% preferred this electronic method of feedback vs. paper format. Overall, 93.6% of student pharmacists felt they were more engaged during the presentations as a result of providing electronic feedback.

SUMMARY

Maximizing our resources by creating an electronic peer evaluation with our current examination software, allowed for an efficient means of obtaining and disseminating peer review that was timely and well-received by students.

Peer Review Research Evaluation in Iran: Strengths, Weaknesses and Suggestions

Background:

Peer based evaluation is a qualitative assessment done in different fields and levels. The aim of this study was to express the results of peer review evaluation in selected Iranian clinical research centers.

Methods:

Four main domains consist of Leadership and governance, Structure, Knowledge products and Impact in thirty Iranian clinical research centers were evaluated based on peer review in 2019. Strengths and weak points with peer‘s suggestions were extracted based on qualitative analysis.

Results:

Governance and impact domains have been more weak points than others. Equipment, facilities, physical space and human resource have been desirable in many research centers, and also there were some good developments in research publication. The most important suggestion was pay more attention to technology in planning, infra-structure and impact levels.

Conclusion:

Review missions of clinical research centers with more emphasis on health impact is necessary to clinical improvement.

Exploring student perceptions and attitude towards various aspects of peer feedback in a pharmacotherapy course

BACKGROUND

The purpose of this study is to gauge pharmacy students' perceptions and attitude towards peer feedback in a pharmacotherapy course.

METHODS

An explanatory sequential mixed-method approach with a 20-item electronic survey and semi-structured interviews were used to collect data from students enrolled in a required pharmacotherapy course at a major public university in the southeast United States. A survey design with descriptive statistics were used for the quantitative part and constant comparative approach was used to analyze qualitative data.

RESULTS

Seventy-three completed surveys (53%) were received (n = 73). Majority of the students (90%) believed that they will be using peer feedback in their future pharmacy careers. Most students (90%) agreed that their peers are competent enough to provide constructive feedback, whereas only 78% believed in their own competency to provide feedback to their peers. Over 81% of students preferred receiving feedback from a peer they have previously worked with. Interviews with five second-year pharmacy students (n = 5) revealed three major themes regarding PharmD students' perception of peer feedback activity, namely, (1) participants' perspectives about the value of peer feedback, (2) learning from peer feedback, and (3) significance of the individuals participating in the peer feedback activities.

CONCLUSIONS

Perceptions of competency, the perceived value of peer feedback and interrelationship among peers are important determinants of effective peer feedback practices. Education and training in techniques and benefits of peer feedback, as well as opportunity to practice the skill can help students promote a positive attitude towards peer feedback.

Comparing faculty evaluations of student journal club presentations with student self- and peer evaluations during advanced pharmacy practice experiences

BACKGROUND AND PURPOSE

Self-evaluation is a critical step in professional development. Peer evaluation may enhance student learning and help peer evaluators recognize their own limitations. However, these evaluations may not accurately assess performance. This study's purpose is to evaluate differences between faculty, self-, and peer evaluations of student journal club (JC) presentations during advanced pharmacy practice experiences (APPEs).

EDUCATIONAL ACTIVITY AND SETTING

Student JC presentations for three APPE sites were identified between May 2015 and April 2018 and included if at least one faculty, self-, and peer evaluation were complete. Overall grades and individual rubric ratings were compared.

FINDINGS

Seventy-four students had complete data sets. The mean overall scores for JC presentations were 82.72%, 86.05%, and 91.01% for faculty, self-, and peer evaluations, respectively. Self-evaluation ratings were significantly higher than faculty on each domain, except for "presentation and communication skills" (mean difference: -0.2, p = .034) and "ability to answer questions" (mean difference: -0.1, p = .247). Linear regression showed a statistically significant relationship between self-evaluation and faculty evaluation ratings for ability to answer questions (beta = 0.5, p < .001). Peer evaluation ratings were significantly higher than faculty and self-evaluations (p < .05).

SUMMARY

Faculty scores on JC presentations completed during APPEs were lower compared to student evaluations of themselves and their peers. Further incorporation of self- and peer evaluation throughout pharmacy school curricula may improve student competence in performing these evaluations. Formal training is needed to improve students' ability to complete self and peer evaluations.

Children's evaluations of deviant peers in the context of science and technology: The role of gender group norms and status

•

Boys negatively evaluate peers who challenge group norms related to computing.

•

Children expect groups to negatively evaluate challenges to science gender norms.

•

Perceptions of group evaluation predict how boys individually evaluate their peers.

Women are drastically underrepresented within computer science, which is in part informed by societal ideas of who can and should belong in the sciences. Less is known about how children evaluate their peers who challenge gendered expectations of who can and should take part in computer science. The current study asked children (N = 213; 110 girls) in middle childhood (M_age = 8.71 years; n = 108) and late childhood (M_age = 10.56 years; n = 105) to evaluate a gender-matched peer who challenged a group norm related to either computer science (male-gendered domain) or biology (less male-gendered domain). Male participants most negatively evaluated a peer who wanted to take part in a biology activity when the rest of the group wanted to do a programming activity. Furthermore, male participants expected their group to negatively evaluate this deviant peer in the programming condition. Mediation analysis revealed that for boys in the computer science condition, perceived group evaluation predicted individual evaluation. Female participants, in contrast, did not negatively evaluate someone who challenged a STEM (science, technology, engineering, and math) peer group norm. This study demonstrates that male peer groups may perpetuate the idea that computer science is for men through negative evaluation of in-group members who challenge those ideas and, in turn, maintain their dominant position as the high-status group. Achieving equity in the computer science field will require a greater understanding of these peer group norms.

Jumpstarting Team Cohesion with Team Activity Debriefings

Although peer evaluations are essential to team-based learning, they can be problematic. Team activity debriefings (TAD) are advantageous because they focus on the team's problem-solving strategies and teamwork skills. Students (N = 100) who used both tools reported that TADs were more helpful in developing team cohesion, helping students understand the characteristics of well-functioning teams, and helping students work better as a team. Peer evaluations were more helpful in evaluating and improving their own contributions to the team. Using both tools may be the best way to foster teamwork skills and hold students accountable for making positive contributions to the team.

Leader Academy: A layered approach to learning leadership

BACKGROUND AND PURPOSE

Leader Academy (LA) is a co-curricular experience for students that builds communication skills, fosters leadership skills, and offers opportunities in professional development. The program couples leadership development during the first year of the curriculum with activities conducted in a team-based learning course.

EDUCATIONAL ACTIVITY AND SETTING

An emotional intelligence (EI) assessment was administered to students during their first year. Students are led through a series of monthly programs utilizing GiANT Worldwide tools and team discussion. LA utilizes faculty, pharmacy residents, and student leaders to serve as facilitators for each team. Students participate in monthly meetings and are assigned team activities within required courses. Peer- and self-assessments are conducted through CATME SMARTER Teamwork program. The EI assessment will be re-administered at the end of LA to assess change in EI. A survey asked students to address the value and relevance of the GiANT Worldwide tools, how often they used the tools, and if they had taught a tool to someone else.

FINDINGS

On average, students scored lower in assertiveness, stress tolerance, emotional expression and independence in comparison to all of the areas assessed. The lowest composite scores were self-expression and stress management. Students indicated that they find the GiANT Worldwide tools valuable and relevant.

SUMMARY

There are specific areas of focus within EI where we can continue to build skills in self-expression and stress management. Students tailor their personal growth in EI by setting SMART goals at the beginning of the program and continually reassess throughout the program.

A Synergy Between Peer Evaluation and Student Coaching in Team-Based Learning: Coach Review of Peer Evaluations Improves Student Acceptance

Peer assessment is an integral component of team-based learning (TBL) and provides a method to foster accountability and team performance. Traditionally, peer assessment in TBL contributes to the final course grade. We assessed the impact of replacing course grading with student's faculty coach review. As a result of this intervention, student self-reported evaluation truthfulness improved from 44 to 76% of the population surveyed. In addition, student self-reported incorporation of peer feedback increased from 18 to 38%. The data suggest that the combination of student coaching and longitudinal review of peer feedback may improve overall effectiveness of the process.

Relationship between peer evaluation and interprofessional self-evaluation in a joint healthcare team-based learning class involving three universities

OBJECTIVE

This study aimed to clarify the relationship between interprofessional self-evaluation and peer evaluation during interprofessional education (IPE) using team-based learning (TBL). We also aimed to clarify differences in interprofessional cooperation between students with high and low peer evaluation scores.

METHODS

In total, 483 students (grades 3-5) from nine faculties at three universities participated in a TBL-based IPE program. The students completed five interprofessional self-evaluation domains (the modified Tsukuba IPE model) before and after IPE. Students also completed peer evaluation after IPE. Students were divided into three groups by peer evaluation scores (low, middle, high), and the post-class self-evaluation scores of these groups were compared using a Kruskal-Wallis test. Multiple regression analysis was also performed. Peer evaluation comments were analyzed using a qualitative inductive method.

RESULTS

Students in the low peer evaluation group had significantly lower scores in the "Regarding participation in group work" domain than students in the high group (P<0.05). Students in the high group received positive comments, such as [good communication] and [working cooperatively], whereas students in the low group were required to improve in two areas: [speaking up more] and [need more communication].

CONCLUSIONS

There was a significant relationship between peer evaluation by team members and self-evaluation for "Regarding participation in group work." Students with high peer evaluation scores participated with active attitudes, whereas students with low scores were considered passive. This study suggested that using peer evaluation may enhance students' professional cooperation by improving their communication and attitudes toward active participation.

Student Perceptions of a Novel Approach to Promote Professionalism Using Peer Evaluation in a Team-Based Learning™ Setting: a Quality Improvement Project

Development of professional behaviors, including communication and interprofessional collaboration, is an important competency during medical school that is difficult to teach in the classroom. We developed an approach to peer evaluation for use in Team-Based Learning that has two novel components: inclusion of a non-numeric sliding scale ranging from "apprentice" to "expert" to assess professional development on a continuum, and incorporation of a structured face-to-face feedback session. Students liked the format of the face-to-face feedback session, felt it gave them an opportunity to better understand their peers' comments, and thought it was more effective than the anonymous written evaluation tool.

A Survey of Health Sciences Faculty Practices and Attitudes Regarding the Peer Feedback Component of Team-Based Learning

BACKGROUND

Peer feedback, an essential component of team-based learning (TBL), fosters accountability among team members. Effective utilization can be advantageous for both students and faculty, although there may be many implementation challenges. Our study objectives were to (1) identify current practices of peer feedback in TBL, (2) identify curricular assessment related to peer feedback, and (3) identify challenges with peer feedback in TBL.

METHODS

A survey link was sent via medical and TBL web boards. The survey asked for both quantitative and qualitative information regarding the peer feedback process. Quantitative results were analyzed using SPSS®. NVivo® was used to identify and code themes in open-ended responses. Both investigators reviewed and agreed upon themes.

RESULTS

Sixty-one TBL users in health professions completed the survey. Multiple health professions were represented with medicine and pharmacy being the most common. Information on current practices showed a wide variety of implementation. Curricular alignment was common. Time and training were the most common challenges and difficulties.

CONCLUSIONS

Meaningful feedback should be considered an important curricular outcome. Specific learning objectives and appropriate assessment strategies should be developed to meet the mission of the program. Students must receive guidance from faculty members on how to improve their skills in providing effective feedback. Overall, faculty saw peer evaluation as valuable. However, the time commitment, lack of student training, and student attitudes make implementation difficult. Future research should focus on best practices to streamline the TBL feedback process and on student training regarding how to provide effective feedback.

A systematic approach to team creation and peer evaluation in a large classroom setting

INTRODUCTION

There is limited data to support a particular method for optimal team creation in pharmacy education. We aimed to implement and evaluate a systematic approach to team creation and compare the impact on team dynamics to teams created via random selection.

METHODS

Two concurrent courses were used to assess team creation methods. Student-specific variables were used for team creation in one course while another course utilized teams created via random allocation. Each course conducted similar peer evaluations to provide feedback and assess team dynamics. A matched pre- and post-course survey assessed changes in student perceptions of team creation and peer evaluation. Student-perceived team effectiveness and individual coursework performance were assessed.

RESULTS

A total of 109 students were enrolled in each of the two courses, with 98% and 93% providing evaluation of team dynamics within the intervention and control methods, respectively. Students perceived better team effectiveness in intervention-created groups in relation to decreased task (p < 0.01), relationship (p < 0.01), and process conflict (p < 0.01), along with increased task attraction (p < 0.01). There was no significant difference in student performance on individual examinations team creation methods (p = 0.17).

CONCLUSIONS

A systematic approach to team creation improved student-perceived team dynamics with no significant impact on coursework scores. A systematic approach to team creation via a web-based platform is feasible in a large classroom setting and may provide an avenue for assessment approaches related to teamwork and team dynamics.

Validity and reliability assessment of a peer evaluation method in team-based learning classes

PURPOSE

Team-based learning (TBL) is increasingly employed in medical education because of its potential to promote active group learning. In TBL, learners are usually asked to assess the contributions of peers within their group to ensure accountability. The purpose of this study is to assess the validity and reliability of a peer evaluation instrument that was used in TBL classes in a single medical school.

METHODS

A total of 141 students were divided into 18 groups in 11 TBL classes. The students were asked to evaluate their peers in the group based on evaluation criteria that were provided to them. We analyzed the comments that were written for the highest and lowest achievers to assess the validity of the peer evaluation instrument. The reliability of the instrument was assessed by examining the agreement among peer ratings within each group of students via intraclass correlation coefficient (ICC) analysis.

RESULTS

Most of the students provided reasonable and understandable comments for the high and low achievers within their group, and most of those comments were compatible with the evaluation criteria. The average ICC of each group ranged from 0.390 to 0.863, and the overall average was 0.659. There was no significant difference in inter-rater reliability according to the number of members in the group or the timing of the evaluation within the course.

CONCLUSION

The peer evaluation instrument that was used in the TBL classes was valid and reliable. Providing evaluation criteria and rules seemed to improve the validity and reliability of the instrument.

Sensitivity to Peer Evaluation and Its Genetic and Environmental Determinants: Findings from a Population-Based Twin Study

Adolescents and young adults are highly focused on peer evaluation, but little is known about sources of their differential sensitivity. We examined to what extent sensitivity to peer evaluation is influenced by interacting environmental and genetic factors. A sample of 354 healthy adolescent twin pairs (n = 708) took part in a structured, laboratory task in which they were exposed to peer evaluation. The proportion of the variance in sensitivity to peer evaluation due to genetic and environmental factors was estimated, as was the association with specific a priori environmental risk factors. Differences in sensitivity to peer evaluation between adolescents were explained mainly by non-shared environmental influences. The results on shared environmental influences were not conclusive. No impact of latent genetic factors or gene-environment interactions was found. Adolescents with lower self-rated positions on the social ladder or who reported to have been bullied more severely showed significantly stronger responses to peer evaluation. Not genes, but subjective social status and past experience of being bullied seem to impact sensitivity to peer evaluation. This suggests that altered response to peer evaluation is the outcome of cumulative sensitization to social interactions.

Associations between maternal negative affect and adolescent's neural response to peer evaluation

Parenting is often implicated as a potential source of individual differences in youths' emotional information processing. The present study examined whether parental affect is related to an important aspect of adolescent emotional development, response to peer evaluation. Specifically, we examined relations between maternal negative affect, observed during parent-adolescent discussion of an adolescent-nominated concern with which s/he wants parental support, and adolescent neural responses to peer evaluation in 40 emotionally healthy and depressed adolescents. We focused on a network of ventral brain regions involved in affective processing of social information: the amygdala, anterior insula, nucleus accumbens, and subgenual anterior cingulate, as well as the ventrolateral prefrontal cortex. Maternal negative affect was not associated with adolescent neural response to peer rejection. However, longer durations of maternal negative affect were associated with decreased responsivity to peer acceptance in the amygdala, left anterior insula, subgenual anterior cingulate, and left nucleus accumbens. These findings provide some of the first evidence that maternal negative affect is associated with adolescents' neural processing of social rewards. Findings also suggest that maternal negative affect could contribute to alterations in affective processing, specifically, dampening the saliency and/or reward of peer interactions during adolescence.