Use of peer assessment in tooth extraction competency

Assessing competency in exodontia: A UK perspective

INTRODUCTION

Assessment ensures that undergraduate students and postgraduate trainees are providing optimal patient care. Numerous assessment methods are currently being used within the United Kingdom. Therefore, the purpose of this review was to provide an overview of the methods used to assess competency in exodontia and compare methods against criteria defined for good assessment.

MATERIALS AND METHODS

A systematic narrative review was carried out. Databases were electronically searched for published studies assessing the competence of undergraduate students and postgraduate trainees in exodontia up to October 2023.The validity, reproducibility, equivalence, feasibility, educational impact, catalytic effect and acceptability of each assessment method was considered.

RESULTS

In this review, 591 undergraduate students (n = 529) and postgraduate dental trainees (n = 62) had their competence in exodontia assessed. Undergraduate students were assessed on simple exodontia with forceps and elevators, whereas postgraduate trainees were assessed on the surgical removal of mandibular third molars. Only one study took place in a simulated dental environment, the remaining (n = 7) took place in a clinical environment. A range of formative and summative assessment methods were identified including structured clinical objective tests (SCOT), self-assessment, peer assessment and logbooks. Common marking criteria such as global rating and checklist scales were also highlighted.

CONCLUSION

This review highlights the lack of evidence to support one form of assessment and the lack of consensus on assessment within the United Kingdom. Further research related to this topic, alongside the opinion of trainees and trainers is required to improve assessments. This will maximise learning opportunities for trainees and translate to better patient care.

Self-perceived preparedness of new dental graduates from a Middle Eastern university

INTRODUCTION

Preparedness for independent clinical practice is a core goal of undergraduate dental education. The aim of this study was to evaluate the self-reported preparedness for the dental practice among new dental graduates from a University in the Middle East.

METHODS

Following ethics approval, a purposive sampling techniques were used to invite newly qualified dental graduates at a Middle Eastern University. A validated research instrument, namely the Dental Undergraduate Preparedness Assessment Scale (DU-PAS), was used to evaluate the self-perceived preparedness of the participants. Data collection was undertaken online using Google Forms and data were analysed to compute the descriptive statistics, reliability and demographic differences among the participants.

RESULTS

A total of 82 participants provided their response yielding a response rate of 68%. The participants included 49 males and 33 females. The overall reliability of DU-PAS was excellent. (α = 0.93). The participants included less than 40% of participants were able independently to prescribe medications, assess orthodontics needs and perform endodontic treatment on multirooted teeth. In addition, more than 65% indicated low confidence in evaluating new dental materials and products, interpreting new research findings and behavioural management of children.

CONCLUSION

The present findings identified strengths and weaknesses of new dental graduates from a Middle Eastern university. The participants were confident in basic clinical skills but lower confidence was reported for complex dental procedures. The findings highlight the learning needs of new dental graduates and can serve to inform the undergraduate curriculum to enhance the preparedness of future cohorts of dental graduates.

Peer-assessment ability of trainees in clinical restorative dentistry: can it be fostered?

Objectives

The study was conducted to investigate whether peer-assessment among dental students at the clinical stage can be fostered and become closer to that of experienced faculty assessors.

Methods

A prospective pilot study was conducted in 2021 at the Faculty of Dentistry, Damascus University. Sixteen 5th year clinical students volunteered to participate in the study. A modified version of the validated Peer Direct Observation of Procedural Skills (Peer-DOPS) assessment form was used together with a grading rubric. Participants undertook peer-assessment on their colleagues across three encounters. The difference between peers and faculty assessment was the main variable.

Results

The mean difference between peers and faculty assessment decreased after each encounter with a significant difference and a medium effect size between the first and third encounters (p = 0.016, d = 0.67). Peer-assessment was significantly higher than faculty, however, the overestimation declined with each encounter reducing the difference between peer- and faculty assessment. Peers’ perception of the educational benefit of conducting assessment was overwhelmingly positive, reporting improvements in their own performance.

Conclusion

This pilot study provides preliminary evidence that dental students assessment ability of their peers can be fostered and become closer to that of experienced faculty assessment with practice and assessment-specific instruction.

An 8-year retrospective survey of assessment in postgraduate dental training in complicated tooth extraction competency

BACKGROUND/PURPOSE

Direct observation of procedural skills (DOPS) has been increasingly used in health education in recent years. This study evaluated the effect of education and trainees' perception of assessment on the clinical skills of postgraduate dental trainees in complicated tooth extraction.

MATERIALS AND METHODS

This study was conducted as a retrospective survey among postgraduate dental trainees learning complicated tooth extraction in Taipei and Linkou Chang Gung Memorial Hospital from 2012 to 2019. Practical skills were assessed using DOPS by trainees and faculty members. Each clinical case included a complicated extraction of a permanent tooth.

RESULTS

A total of 69 participants (26 men and 43 women, average age = 26.49 years, range = 24-34 years) were included in this study. Within the survey cohort, faculty assessments scored significantly higher than did trainees' self-assessments in each complicated tooth extraction procedure, with no difference between both sexes. The higher-performing trainees tended to underrate their performance much more than did the lower-performing trainees. More than 40% of the trainees evaluated themselves as having "poor capability" in some invasive surgical procedures, even though their actual performance was not lower than that of those who evaluated themselves as having adequate or good capability.

CONCLUSION

Self-assessment skills should be developed with more practice and experience. We hope that these findings can guide the planning of faculty development programs for clinical instructors, particularly the new cohort of faculty who will succeed the rapidly retiring boomer generation in the next 10 years.

Dental student self-assessment of a medical history competency developed by Oral and Maxillofacial Surgery Faculty

PURPOSE

To develop a standardized competency exam to evaluate dental student knowledge of patients with complex medical histories. Analysis was performed to determine if there is a difference between dental student self-assessment of the exam compared to oral surgery faculty. The overall goal is to enhance student comprehension of advanced medical patients in the pre-doctoral oral and maxillofacial surgery clinic and make changes based on student responses.

METHODS

The exam took place in a simulation laboratory containing two-way mirrors where the student could not see the evaluator. Three standardized patients (trained actors) were given different medical history scenarios to learn prior to the exam. Students were randomly assigned to interview one patient. The graders consisted of senior and junior level faculty. In real time (live) randomized manner, faculty assessed each student using a standardized rubric to assign a score of 1 to 4. Immediately afterwards, students assessed their own performance using the same scale.

RESULTS

Students gave themselves significantly higher grades (P < 0.001) on the exam (10.7 ± 0.12) compared to the faculty (9.2 ± 0.17). There was no significant difference in student (P = 0.16) or faculty assessment (P = 0.29) between the three different medical scenarios. There was no significant difference between faculty in assessment of student performances (P = 0.16).

CONCLUSION

These results suggested that students overestimated their performance compared to the faculty assessment. This could be due to that students assessed their performance based on memory and self-confidence; that exceeds their expectations. Internal calibration of student performances and faculty grading using standardized criteria may produce more accurate and reliable outcomes in student scoring.

Analysis of dental students' written peer feedback from a prospective peer assessment protocol

INTRODUCTION

Peer assessment and feedback is encouraged to enhance students' learning. The aim of this study was to quantitatively and qualitatively analyse pre-clinical and clinical dental students' written peer feedback provided as part of a continuous, formative and structured peer assessment protocol.

MATERIALS AND METHODS

A total of 309 Year-2 and Year-5 dental students were invited to participate in a peer assessment and peer feedback protocol. Consenting volunteer students were trained to observe each other whilst working in the skills laboratory (Year-2) and in the dental clinic (Year-5). Subsequently, they followed a structured protocol of peer assessment and peer feedback using specially designed work-based forms during a complete academic year. The content of their written feedback was coded according to the UK General Dental Council domain, sign (positive or negative), specificity (task specific or general), and grouped into themes.

RESULTS

A total of 108 participants (40 Year-2 and 68 Year-5) completed 1169 peer assessment work-based forms (516 pre-clinical and 653 clinical); 94% contained written feedback. The large majority (82%) of Year-2 feedback represented the clinical domain, 89% were positive, 77% were task specific, and they were grouped into 14 themes. Year-5 feedback was related mostly to Management and Leadership (37%) and Communication (32%), 64% were positive, 75% task specific, and they were clustered into 24 themes.

DISCUSSION

The content of the feedback showed notable differences between Year-2 and Year-5 students. Senior students focused more on Communication and Management and Leadership skills, whilst juniors were more concerned with clinical skills. Year-5 students provided 13% negative feedback compared to only 2% from Year-2. Regulatory focus theory is discussed to explain these differences. Both groups provided peer feedback on a wide and different range of themes. However, four themes emerged in both groups: efficiency, infection control, time management and working speed.

CONCLUSION

A structured peer assessment framework can be used to guide pre-clinical and clinical students to provide peer feedback focused on different domains, and on contrasting signs and specificities. It can also present an opportunity to complement tutors' feedback.

Dental students' peer assessment: a prospective pilot study

INTRODUCTION

Peer assessment is increasingly used in health education. The aims of this study were to evaluate the reliability, accuracy, educational impact and student's perceptions of undergraduate pre-clinical and clinical dental students' structured and prospective Peer assessment and peer feedback protocol.

MATERIALS AND METHODS

Two Direct Observation of Procedural Skills (DOPS) forms were modified for use in pre-clinical and clinical peer assessment. Ten year two dental students working in a phantom-heads skills laboratory and 16-year five dental students attending a comprehensive care clinic piloted both peer DOPS forms. After training, pairs of students observed, assessed and provided immediate feedback to each other using their respective peer DOPS forms as frameworks. At the end of the 3-month study period, students anonymously provided their perceptions of the protocol.

RESULTS

Year 2 and year 5 students completed 57 and 104 peer DOPS forms, respectively. The generalizability coefficient was 0.62 for year 2 (six encounters) and 0.67 for year 5 (seven encounters). Both groups were able to differentiate amongst peer-assessed domains and so detect improvement in peers' performance over time. Peer DOPS scores of both groups showed a positive correlation with their mean end-of-year examination marks (r ≥ 0.505, P ≥ 0.051) although this was not statistically significant. There was no difference (P ≥ 0.094) between the end-of-year examination marks of the participating students and the rest of their respective classes. The vast majority of both groups expressed positive perceptions of the piloted protocol.

DISCUSSION

There are no data in the literature on the prospective use of peer assessment in the dental undergraduate setting. In the current study, both pre-clinical and clinical students demonstrated the ability to identify those domains where peers performed better, as well as those which needed improvement. Despite no observable educational impact, most students reported positive perceptions of the peer DOPS protocol.

CONCLUSIONS

The results of this pilot study support the need for and the potential benefit of a larger- and longer-term follow-up study utilising the protocol.