Novices can assess microsurgery performance, and this is enhanced using the Manual Suture Parameters for Training and Assessment (M-SParTA) scoring system

This study explores how novices could effectively evaluate the quality of microsurgical suturing. That would be enhanced with using a novel Manual Suture Parameters for Training and Assessment (M-SParTA), which supported novices with guidance on the objective parameters, in order to increased the accuracy of scoring ability. We also propose the following initial framework to train novices in microsuturing using a standardised task: 1) Exposure; 2) Assessment; 3) Hands-on and self-assessment. The independent learning cycle with targeted supervision provides novices with greater autonomy and a less stressful environment that could enhance skills training.

Objective Structured Assessment of technical skill (OSATS) in the Surgical Skills and Technology Elective Program (SSTEP): Comparison of peer and expert raters

BACKGROUND

A major challenge with Competency Based Medical Education (CBME) is that of increased assessment burden on faculty. To reduce this burden, the accuracy and reliability of peer-assessment for surgical skills requires further exploration.

METHODS

Forty-two second year medical students were video recorded while performing a simple interrupted suture and an instrument tie. Four novice raters underwent a short training session on the use of the Objective Structured Assessment of Technical Skills (OSATS) checklists. Videos of the suturing task were then independently assessed by the four novice raters and two expert raters on two occasions. Agreement between novice and expert rater scores was calculated using the intraclass correlation coefficient (ICC).

RESULTS

For both simple interrupted suturing (ICC = 0.78, CI = 0.66-0.86, p < 0.001) and instrument ties (ICC = 0.87, CI = 0.80-0.92, p < 0.001), there was good agreement between novice and expert raters.

CONCLUSIONS

Novice raters can be taught to use the OSATS checklists to assess peers on simple suturing and instrument tying tasks.

Pursue today and assess tomorrow - how students' subjective perceptions influence their preference for self- and peer assessments

BACKGROUND

Alternative assessments engage students in the assessment process to improve both short- and long-term outcomes by developing their judgments and responsibility about their own learning, and that of their peers. In this study, we investigated students' perception towards self- and peer-assessment, their objectivity and impact on students' learning.

METHODS

The study was conducted at the Medical University of Vienna. Attitudes of second year undergraduate medical students towards self- and peer-assessment, and their objectivity, appropriateness, and the impact of these assessments on students' learning activities, was inquired using a self-developed questionnaire.

RESULTS

Four hundred twenty-three students participated in this study. Self-assessment was found more appropriate method to assess students' knowledge. Most of students agreed that peer-assessment is not objective (M = - 0.07). Majority of students evaluated that peer assessment has no or little impact on their active and passive learning (M = - 0.23, - 0.35), on the other hand self-assessment was reported as a helpful tool for gaining long-term knowledge (M = 0.13) and following the content of courses (M = 0.16).

CONCLUSION

Based on our results, students' perspective on peer assessment were negative, on the whole, students had positive attitudes towards self-assessment and negative attitudes towards peer-assessment. This study also determined that self-assessment leads to the promotion of students' learning.

Evaluation of a peer- and self-grading process for clinical writing assignments

OBJECTIVES

As class sizes in pharmacy education increase, faculty must develop new assessment strategies for essay writing assignments. This study evaluated accuracy and student perceptions of an innovative grading process that utilizes both peer- and self-assessment.

METHODS

Four SOAP note sessions were evaluated. Each session included four activities: a writing workshop, assessment workshop, and reflection session. For each assessment workshop students scored their note and a blinded peer's note using a grading form, facilitated by a faculty-led discussion. In a subsequent reflection session, students reviewed their peer- and self-assigned grades and could petition for faculty review if desired. The average self-, peer-, and final-grades were compared for each of the four SOAP note sessions using ANOVA. After the fourth session, students completed an anonymous 10-question Likert-scale survey regarding their perceptions of the process and three open-response questions. Survey results were analyzed with descriptive statistics.

RESULTS

Approximately 140 students participated. No difference was found between the average self-, peer-, and final-grades for all four sessions (p > 0.05). The survey response rate was 65% (91/140). Survey questions were grouped into three themes. The majority of students either strongly agreed or agreed that sessions were well organized and effective (≥84%), assessment workshops enhanced learning (≥68%), and the scoring method was fair (≥72%). The lowest score (mean 2.53 on a 4-point scale) reflected satisfaction with peer-provided feedback. When asked what they liked most, respondents most commonly cited that faculty-led review and discussion enhanced clinical knowledge.

IMPLICATIONS

The combination of a peer- and self-assessment process was accurate, well-received, and can be used to decrease faculty workload.

Experiences of using an OSCE protocol in clinical examinations of nursing students - A comparison of student and faculty assessments

Peer-assessment in nursing education using an OSCE protocol is an increasing educational activity that complements that of teachers. However, little is known about how students' and teachers' assessments correspond. The study aimed to compare OSCE assessments made by student examiners and faculty examiners during examinations of clinical skills in undergraduate nursing education. Four cohorts of third-year nursing students participated between 2014 and 2016. The students underwent a clinical examination of the management of central venous catheters and totally implantable venous access devices. Students who performed the examinations were observed both by a faculty examiner and student examiner. Both observers used the same OSCE protocol for the assessment but independently. The OSCE protocols from both faculty and student examiners were reviewed and compared. Total agreement between the student and faculty examiner was reached in 127 of 135 (94%) paired protocols. The level of agreement was substantial with a kappa value of 0.79 (95% CI 0.65-0.93). The conclusion was that the level of agreement between student and faculty examiners was high when using an OSCE protocol in clinical examinations of two different clinical skill tasks. The structured checklist (OSCE protocol) was easy to use for the student examiners despite the lack of experience or training in advance.

Can student self-ratings be compared with peer ratings? A study of measurement invariance of multisource feedback

Measurement invariance is a prerequisite for comparing measurement scores from different groups. In medical education, multi-source feedback (MSF) is utilized to assess core competencies, including the professionalism. However, little attention has been paid to the measurement invariance of assessment instruments; that is, whether an instrument holds the same meaning across different rater groups. To examine the measurement invariance of the National Taiwan University professionalism MSF (NTU P-MSF) in order to determine whether medical students' self-rating can be compared to their peers' rating. An eight-factor model was specified for confirmatory factor analysis to examine the construct validity of the NTU P-MSF. Cronbach's alpha was computed for the items of each domain to evaluate internal consistent reliability. The same eight-factor model was used for multi-group confirmatory factor analyses. Four hierarchical models were specified to test configural (i.e., identical factor-item relationship), metric (i.e., identical factor loadings), scalar (i.e., identical intercepts), and error variance across self-rating and peer rating groups. One hundred and twenty second-year medical students from weekly discussion groups conducted as part of a medical professionalism course agreed to use the NTU P-MSF to assess themselves or their discussion group peers. NTU P-MSF assessment scores were a good fit for the eight-factor model among self group and peer group. The Cronbach's alpha coefficients of students' NTU P-MSF scores and peers' scores ranged from 0.76 to 0.89 and 0.84 to 0.91, respectively indicating that the NTU P-MSF scores also have good internal consistent reliability between both groups. In addition, same factor structure and similar factor loadings and intercepts of NTU P-MSF scores between both groups indicate that NTU P-MSF scores had configural, metric, and scalar invariance. Thus, students' self-assessments and peer assessments can be compared in terms of the constructs of NTU P-MSF scores, change in NTU P-MSF scores, and its factor scores. This study demonstrates how to investigate the measurement invariance of a professionalism MSF and contributes to the discussion on self- and peer assessment in medical education.

Does self-reflection and peer-assessment improve Saudi pharmacy students' academic performance and metacognitive skills?

BACKGROUND

The patient-centered focus of clinical pharmacy practice which demands nuanced application of specialized knowledge and skills targeted to meeting patient-specific therapeutic needs warrant that the training strategy used for PharmD graduates must empower with the ability to use the higher level cognitive processes and critical thinking effectively in service delivery. However, the historical disposition to learning in the Middle East and among Saudi students appeared heavily focused on rote memorization and recall of memorized facts.

OBJECTIVES

To assess the impact of active pedagogic strategies such as self-reflection and peer assessment on pharmacy students' academic performance and metacognitive skills, and evaluate students' feedback on the impact of these active pedagogic strategies on their overall learning experience.

METHOD

An exploratory prospective cohort study was conducted among 4th year students at the College of Clinical Pharmacy, King Faisal University, Saudi Arabia to assess the impact of self-reflection and peer-assessment in a semester-wide assessment tasks in two compulsory first semester 4th year courses (Therapeutics-3 and Pharmacoeconomics). An end-of-course evaluation survey with a pre-tested 5-item open-ended questionnaire was also conducted to evaluate students' feedback on the impact of active pedagogic strategies on their overall learning experience.

RESULT

Male students (study group) constituted 40.7% of the cohort while 59.3% were females (control group) with mean ± SD age of 23.2 ± 5.6 and 22.1 ± 4.9 years respectively. The mean ± SD scores for quizzes, mid-term and final exams, and the overall percentage pass were significantly higher in the study group for both courses (P < 0.001). The majority of the students in the study group opined that the exposure to active pedagogic strategies enabled them to improve their use of critical thinking, facilitated deeper engagement with their learning and improved their clinical decision-making and discussion skills.

CONCLUSION

The use of active pedagogic strategies such as self-reflection and peer-assessment appeared to significantly improve examination performance, facilitate deep and constructive engagement with learning and fostered students' confidence in the use of critical thinking and clinical decision-making.

Nursing students' experiences with the use of authentic assessment rubric and case approach in the clinical laboratories

BACKGROUND

One current challenge for nurse educators is to examine effective nursing assessment tools which integrate nursing knowledge into practice. Authentic assessment allows nursing students to apply knowledge to real-life experiences. Contextualized cases have engaged students for preparation of diverse clinical situations and develop critical thinking skills.

AIM

This study aimed to explore nursing students' experiences and learning outcomes with the use of an authentic assessment rubric and a case approach.

METHODS

An exploratory qualitative approach using focus-group discussions and an open-ended survey was adopted. Sixteen nursing students participated in three focus-group discussions and 39 nursing students completed an open-ended survey.

RESULTS

Nursing students noted that an authentic assessment rubric with a case approach provided clarity for their learning goals; built confidence; developed knowledge, skill competencies and critical thinking skills; increased awareness of caring attributes and communication skills; and enriched and extended learning through self-, peer- and teacher-assessments.

CONCLUSIONS

These findings provide rich insights for nurse educators and curriculum developers in the use of an authentic assessment rubric and a case approach in nursing education.

Peer-assessment of medical communication skills: the impact of students' personality, academic and social reputation on behavioural assessment

OBJECTIVE

Peer-assessment of communication skills may contribute to mastery of assessment criteria. When students develop the capacity to judge their peers' performance, they might improve their capacity to examine their own clinical performance. In this study peer-assessment ratings are compared to teacher-assessment ratings. The aim of this paper is to explore the impact of personality and social reputation as source of bias in assessment of communication skills.

METHODS

Second year students were trained and assessed history taking communication skills. Peers rated the students' personality and academic and social reputation.

RESULTS

Peer-assessment ratings were significantly correlated with teacher-ratings in a summative assessment of medical communication. Peers did not provide negative ratings on final scales but did provide negative ratings on subcategories. Peer- and teacher-assessments were both related to the students' personality and academic reputation.

CONCLUSION

Peer-assessment cannot replace teacher-assessment if the assessment should result in high-stake decisions about students. Our data do not confirm the hypothesis that peers are overly biased by personality and reputation characteristics in peer-assessment of performance.

PRACTICE IMPLICATIONS

Early introduction of peer-assessment in medical education would facilitate early acceptance of this mode of evaluation and would promote early on the habit of critical evaluation of professional clinical performance and acceptance of being evaluated critically by peers.