A Novel Approach to Assessing Professionalism in Preclinical Medical Students Using Multisource Feedback Through Paired Self- and Peer Evaluations

Peer overmarking and insufficient diagnosticity: the impact of the rating method for peer assessment

The present study explores two rating methods for peer assessment (analytical rating using criteria and comparative judgement) in light of concurrent validity, reliability and insufficient diagnosticity (i.e. the degree to which substandard work is recognised by the peer raters). During a second-year undergraduate course, students wrote a one-page essay on an air pollutant. A first cohort (N = 260) relied on analytical rating using criteria to assess their peers' essays. A total of 1297 evaluations were made, and each essay received at least four peer ratings. Results indicate a small correlation between peer and teacher marks, and three essays of substandard quality were not recognised by the group of peer raters. A second cohort (N = 230) used comparative judgement. They completed 1289 comparisons, from which a rank order was calculated. Results suggest a large correlation between the university teacher marks and the peer scores and acceptable reliability of the rank order. In addition, the three essays of substandard quality were discerned as such by the group of peer raters. Although replication research is warranted, the results provide the first evidence that, when peer raters overmark and fail to identify substandard work using analytical rating with criteria, university teachers may consider changing the rating method of the peer assessment to comparative judgement.

Multisource feedback: an overview of its use and application as a formative assessment

Multisource feedback (MSF), often termed 360-degree feedback, is a formative performance assessment in which data about an individual's observable workplace behaviors are collected through questionnaires from those interacting with the individual; data are aggregated for anonymity and confidentiality; the aggregated data, along with self-assessment if available, are provided to the individual; and the recipient meets with a trusted individual to review the data and develop an action plan. It is used along the continuum of medical education. This article provides an overview of MSF's utility, its evidence base and cautions.

Formative peer assessment in higher healthcare education programmes: a scoping review

OBJECTIVES

Formative peer assessment focuses on learning and development of the student learning process. This implies that students are taking responsibility for assessing the work of their peers by giving and receiving feedback to each other. The aim was to compile research about formative peer assessment presented in higher healthcare education, focusing on the rationale, the interventions, the experiences of students and teachers and the outcomes of formative assessment interventions.

DESIGN

A scoping review.

DATA SOURCES

Searches were conducted until May 2019 in PubMed, Cumulative Index to Nursing and Allied Health Literature, Education Research Complete and Education Research Centre. Grey literature was searched in Library Search, Google Scholar and Science Direct.

ELIGIBILITY CRITERIA

Studies addressing formative peer assessment in higher education, focusing on medicine, nursing, midwifery, dentistry, physical or occupational therapy and radiology published in peer-reviewed articles or in grey literature.

DATA EXTRACTIONS AND SYNTHESIS

Out of 1452 studies, 37 met the inclusion criteria and were critically appraised using relevant Critical Appraisal Skills Programme, Joanna Briggs Institute and Mixed Methods Appraisal Tool tools. The pertinent data were analysed using thematic analysis.

RESULT

The critical appraisal resulted in 18 included studies with high and moderate quality. The rationale for using formative peer assessment relates to giving and receiving constructive feedback as a means to promote learning. The experience and outcome of formative peer assessment interventions from the perspective of students and teachers are presented within three themes: (1) organisation and structure of the formative peer assessment activities, (2) personal attributes and consequences for oneself and relationships and (3) experience and outcome of feedback and learning.

CONCLUSION

Healthcare education must consider preparing and introducing students to collaborative learning, and thus develop well-designed learning activities aligned with the learning outcomes. Since peer collaboration seems to affect students' and teachers' experiences of formative peer assessment, empirical investigations exploring collaboration between students are of utmost importance.

Evaluating the effectiveness of undergraduate clinical education programs

Medical schools should use a variety of measures to evaluate the effectiveness of their clinical curricula. Both outcome measures and process measures should be included, and these can be organized according to the four-level training evaluation model developed by Donald Kirkpatrick. Managing evaluation data requires the institution to employ deliberate strategies to monitor signals in real-time and aggregate data so that informed decisions can be made. Future steps in program evaluation includes increased emphasis on patient outcomes and multi-source feedback, as well as better integration of existing data sources.

Alignment of Ethics Curricula in Medical Education: A Student Perspective

Issue: Although there is consensus on the importance of including ethics in the medical school curriculum, there is wide variation in how this topic is taught. Recent literature also questions the effectiveness of current ethical teaching methods in changing student attitudes and future behavior. Furthermore, from the student perspective, there is a marked disconnect between the stated importance of and lack of effort in ethics courses. Evidence: Applying a student perspective of the hidden curriculum, as well as reviewing and applying insight from the available literature, we advocate for alignment of instructional design, content, and assessments. This article provides specific recommendations to increase student engagement in ethics courses and concludes by discussing whether a lack of engagement is attributable to intrinsic qualities of medical students in addition to pedagogical technique and educational setting and culture. Implications: This article has practical suggestions for medical educators to improve their ethics courses, leading to more well-rounded and thoughtful physicians.

Assessing Professionalism in Medicine - A Scoping Review of Assessment Tools from 1990 to 2018

BACKGROUND

Medical professionalism enhances doctor-patient relationships and advances patient-centric care. However, despite its pivotal role, the concept of medical professionalism remains diversely understood, taught and thus poorly assessed with Singapore lacking a linguistically sensitive, context specific and culturally appropriate assessment tool. A scoping review of assessments of professionalism in medicine was thus carried out to better guide its understanding.

METHODS

Arksey and O'Malley's (2005) approach to scoping reviews was used to identify appropriate publications featured in four databases published between 1 January 1990 and 31 December 2018. Seven members of the research team employed thematic analysis to evaluate the selected articles.

RESULTS

3799 abstracts were identified, 138 full-text articles reviewed and 74 studies included. The two themes identified were the context-specific nature of assessments and competency-based stages in medical professionalism.

CONCLUSIONS

Prevailing assessments of professionalism in medicine must contend with differences in setting, context and levels of professional development as these explicate variances found in existing assessment criteria and approaches. However, acknowledging the significance of context-specific competency-based stages in medical professionalism will allow the forwarding of guiding principles to aid the design of a culturally-sensitive and practical approach to assessing professionalism.

The utilization of peer feedback during collaborative learning in undergraduate medical education: a systematic review

BACKGROUND

Peer evaluation can provide valuable feedback to medical students, and increase student confidence and quality of work. The objective of this systematic review was to examine the utilization, effectiveness, and quality of peer feedback during collaborative learning in medical education.

METHODS

The PRISMA statement for reporting in systematic reviews and meta-analysis was used to guide the process of conducting the systematic review. Evaluation of level of evidence (Colthart) and types of outcomes (Kirkpatrick) were used. Two main authors reviewed articles with a third deciding on conflicting results.

RESULTS

The final review included 31 studies. Problem-based learning and team-based learning were the most common collaborative learning settings. Eleven studies reported that students received instruction on how to provide appropriate peer feedback. No studies provided descriptions on whether or not the quality of feedback was evaluated by faculty. Seventeen studies evaluated the effect of peer feedback on professionalism; 12 of those studies evaluated its effectiveness for assessing professionalism and eight evaluated the use of peer feedback for professional behavior development. Ten studies examined the effect of peer feedback on student learning. Six studies examined the role of peer feedback on team dynamics.

CONCLUSIONS

This systematic review indicates that peer feedback in a collaborative learning environment may be a reliable assessment for professionalism and may aid in the development of professional behavior. The review suggests implications for further research on the impact of peer feedback, including the effectiveness of providing instruction on how to provide appropriate peer feedback.

Why Pull the Arrow When You Cannot See the Target? Framing Professionalism Goals in Medical Education

Professionalism is essential for a successful physician-patient relationship and widely acknowledged as an intrinsic and important component of medical education for learners at all levels, from medical school to residency to continuing medical education. The problem is defining the educational end points for learners and then determining how to assess these outcomes. This Invited Commentary focuses on what medical educators can and should do to refine the vision of professionalism in medical education. The authors propose a multistep process in which learners, educators, and the public are engaged in articulating clearly and definitively the end points of professionalism education.

Peer Feedback on Collaborative Learning Activities in Veterinary Education

Collaborative learning activities are an increasingly prominent feature of veterinary curricula that have been redesigned to achieve competency-based graduate learning outcomes. This evolution challenges the traditional individualistic approach to veterinary education and necessitates revisions to assessment and feedback practices to ensure constructive alignment. Peer feedback has been widely reported in the medical education literature as a teaching intervention in collaborative learning settings, with learning gains reported for students who receive and provide peer feedback. In this setting, peer feedback has been demonstrated to provide valuable formative feedback on professional behaviors and skills. However, there are very few such reports in the veterinary education literature to date. Barriers to the introduction of this approach can include teacher and student perceptions, and concerns around validity and reliability. This review aimed to provide an overview of current evidence regarding peer feedback on collaborative learning activities in higher education, and to explore opportunities and challenges for the introduction of peer feedback in the context of veterinary education. We contend that early and repeated provision of formative peer feedback can provide an opportunity to scaffold the development of crucial core competencies within veterinary education, including the self-regulated learning skills required to work in collaborative teams, and interpret and act on feedback.