The relationship between formative and summative assessment of undergraduates in oral surgery

Clinical experience improves academic outcomes in dental Oral surgery education

INTRODUCTION

This paper aimed to determine if the level of a dental student's clinical experience in oral surgery influences the student's oral surgery academic outcomes.

MATERIALS AND METHODS

The oral surgery clinical activity and academic outcomes for all students who completed their dental training in 2017, 2018, 2019 and 2020 from the University of Sydney were analysed for correlation.

RESULTS

The clinical activity performed by 313 students was recorded. There was a weak, but statistically significant, Pearson correlation between total dental extractions and academic outcomes (r = .243, p = <.001). The total number of dental extractions performed was the only statistically significant variable on academic outcomes with linear regression analysis (β = .227, p = .005; model R2  = .077). The increase of a student's clinical experience from less than 45 to more than 65 dental extractions raised their oral surgery academic results by an average of 6.4%. There were no academic benefits to earlier clinical experience.

CONCLUSION

An increase in experience with dental extractions resulted in an increase in oral surgery academic outcomes, plateauing at 65 dental extractions. There was no academic advantage to dental student's having earlier clinical experience.

Dental students' and faculty perception of online exams with e-invigilation in Cyprus

INTRODUCTION

During the recent pandemic, e-learning and e-assessment methods have been implemented worldwide, providing opportunities for further implementation in the dental curriculum. This study aims to research the dental students' and dental faculty's perceptions of online exams with e-invigilation.

MATERIALS AND METHODS

Online questionnaires were developed and delivered, after three semesters of online exams, to all students and faculty. Descriptive statistics were performed, and answers were grouped into Principal Components (PC) using Principal Component Analysis (PCA). Statistical significance was set at p < .05.

RESULTS

Two-hundred and sixty dental students (83.7%) and 24 dental faculty members (63.1%) answered the online questionnaires. PCA of students' responses revealed 4 PC, 'University support to students', 'Comparison between online and face-to-face exams', 'Preparation for the online exams' and 'Attitudes towards the technology used for the online exams'. PCA of faculty responses revealed 5 PC: 'Comparison between online and face-to-face exams', 'University support to faculty', 'Faculty attitudes towards the exam procedures', 'Human factors associated with the exam procedures' and 'Exam invigilation'. The overall satisfaction was high for both staff and students (higher for students and female staff). Students with previous experience in online exams scored more positively than first-year students. University support, process-related stress and e-invigilation were highlighted.

CONCLUSIONS

The overall satisfaction with the e-exams was high, despite the technical problems, time-consuming processes and related stress. University support (training, IT support and resources) and mock exams emerged as important elements of online exams, as was the e-invigilation, which students perceived as efficient and non-intrusive.

Early Evaluation of a New French Surgery Course in the Best Practice of Dealing With Major Incidents and Mass Casualty Events

OBJECTIVE

The main objective of this study is to evaluate the impact of a nationwide 5-month course aimed to prepare surgeons for Major Incidents through the acquisition of key knowledge and competencies. Learners' satisfaction was also measured as a secondary objective.

DESIGN

This course was evaluated thanks to various teaching efficacy metrics, mainly based on Kirkpatrick's hierarchy in medical education. Gain in knowledge of participants was evaluated by multiple-choice tests. Self-reported confidence was measured with 2 detailed pre and post training questionnaires.

SETTING

Creation in 2020 of a nationwide, optional and comprehensive Surgical Training in War and Disaster Situation as part of the French surgery residency program. In 2021, data was gathered regarding the impact of the course on participants' knowledge and competencies.

PARTICIPANTS

The study included 26 students in the 2021 cohort (13 residents and 13 practitioners).

RESULTS

Mean scores were significantly higher in the post-test compared to the pre-test, showing significant increase in participants' knowledge during the course: 73,3% vs. 47,3% respectively (p ≤ 0.001). Average learners' confidence scores to perform technical procedures showed at least a +1-point increase on the Likert scale for 65% of items tested (p ≤ 0.001). 89% of items showed at least a +1-point increase on the Likert scale when it came to average learners' confidence score on dealing with complicated situations (p ≤ 0.001). Our post-training satisfaction survey showed that 92% of all participants have noticed the impact of the course on their daily practice.

CONCLUSION

Our study shows that the third level of Kirkpatrick's hierarchy in medical education was reached. This course therefore appears to be meeting the objectives set by the Ministry of Health. Being only 2 years old, it is on the road to gathering momentum and further development.

Formative Objective Structured Clinical Examinations (OSCEs) as an Assessment Tool in UK Undergraduate Medical Education: A Review of Its Utility

The Objective Structured Clinical Examination (OSCE) is a globally established clinical examination; it is often considered the gold standard in evaluating clinical competence within medicine and other healthcare professionals' educations alike. The OSCE consists of a circuit of multiple stations testing a multitude of clinical competencies expected of undergraduate students at certain levels throughout training. Despite its widespread use, the evidence regarding formative renditions of the examination in medical training is highly variable; thus, its suitability as an assessment has been challenged for various reasons. Classically, Van Der Vleuten's formula of utility has been adopted in the appraisal of assessment methods as means of testing, including the OSCE. This review aims to provide a comprehensive overview of the literature surrounding the formative use of OSCEs in undergraduate medical training, whilst specifically focusing on the constituents of the equation and means of mitigating factors that compromise its objectivity.

Evidence for commonly used teaching, learning and assessment methods in contact lens clinical skills education

INTRODUCTION

Evidence based practice is now an important part of healthcare education. The aim of this narrative literature review was to determine what evidence exists on the efficacy of commonly used teaching and learning and assessment methods in the realm of contact lens skills education (CLE) in order to provide insights into best practice. A summary of the global regulation and provision of postgraduate learning and continuing professional development in CLE is included.

METHOD

An expert panel of educators was recruited and completed a literature review of current evidence of teaching and learning and assessment methods in healthcare training, with an emphasis on health care, general optometry and CLE.

RESULTS

No direct evidence of benefit of teaching and learning and assessment methods in CLE were found. There was evidence for the benefit of some teaching and learning and assessment methods in other disciplines that could be transferable to CLE and could help students meet the intended learning outcomes. There was evidence that the following teaching and learning methods helped health-care and general optometry students meet the intended learning outcomes; clinical teaching and learning, flipped classrooms, clinical skills videos and clerkships. For assessment these methods were; essays, case presentations, objective structured clinical examinations, self-assessment and formative assessment. There was no evidence that the following teaching and learning methods helped health-care and general optometry students meet the intended learning outcomes; journal clubs and case discussions. Nor was any evidence found for the following assessment methods; multiple-choice questions, oral examinations, objective structured practical examinations, holistic assessment, and summative assessment.

CONCLUSION

Investigation into the efficacy of common teaching and learning and assessment methods in CLE are required and would be beneficial for the entire community of contact lens educators, and other disciplines that wish to adapt this approach of evidence-based teaching.

Association Between Formative Assessment and Academic Performance for Undergraduate Medical Students in a Chinese Clinical Skills Training Course

BACKGROUND Previous research suggests that formative assessment (FA) enhances learning outcomes, but few studies have evaluated its impact on clinical skills training in China. We conducted this study in a clinical skills integral curriculum to further explore the educational value of FA. MATERIAL AND METHODS Sixty undergraduates from the Second Clinical Medical School of the Southern Medical University in 2016 were selected as the experimental group (consecutive FA), and 50 undergraduates in 2015 were selected as the control term (only final summative assessment, SA). Undergraduates in the FA group completed the after-class questionnaire at each lesson. Teachers, teaching content, assessment objectives, and topics are the same in both groups. RESULTS The results of single-factor covariance (ANCOVA) analysis and Cochran-Mantel-Haenszel (CMH) analysis demonstrated that students of the FA group obtained better performance and higher success rates in summative examination than in the SA group. The students with relatively poor grades benefited more from FA, while the performance of students with higher grades was similar between the FA group and SA group. According to the results of questionnaire for students, the satisfaction of students with the course increased gradually, from 84.4% to 93.0%. CONCLUSIONS Proper use of FA is associated with better learning outcomes for students, especially for those with poorer grades. Our results, together with previous research, indicated that the use of FA may be of great benefit to students' academic performance and satisfaction with the clinical skills training curriculum.

Introducing a comprehensive high-stake online exam to final-year dental students during the COVID-19 pandemic and evaluation of its effectiveness

BACKGROUND

Dental education involves teaching and assessing the acquisition of verifiable domains that require superior psychomotor, communication, and cognitive skills. Evolving technologies and methods of assessment could enhance student learning environment and improve tutor assessment experience.

OBJECTIVE

The aim of this study was to introduce the application of a comprehensive high-stakes online exam to final-year dental students during the COVID-19 pandemic and evaluate its effectiveness.

DESIGN

A high-stakes exam was introduced and implemented online to the final-year dental students prior to their graduation. The exam consisted of four components: MEQs, MCQs, OSCE and an oral exam. The exam and invigilation were conducted using Blackboard and MS Teams programs. Stakeholders' views of the exam were obtained using two tailored surveys, one for students and another for faculty; both included closed- and open-ended questions.

RESULTS

The exam was run successfully without untoward events. Both students and staff were satisfied with the online exam with the latter being more satisfied than the former. Students with previous experience in online learning system were more satisfied with the online exam compared with those with less experience (p < 0.05). The main issues raised by students' satisfaction with the exam were: inadequacy of time for the MEQ part, prevention of back tracking in the MCQ part and minor technological issues, whereas those raised by faculty members were increased time required to complete the exam setup and grading compared to the paper-based exam and minor technological issues.

CONCLUSIONS

A newly introduced, multi-format, online high-stakes exam was implemented successfully to final-year dental students with minor technological issues and good satisfaction by students and staff alike.

Learning curve of intraoral scanning by prosthodontic residents

STATEMENT OF PROBLEM

The use of intraoral scanning (IOS) to replace conventional impressions has become popular. However, the learning time required to use these devices effectively is unknown.

PURPOSE

The purpose of this in vitro study was to investigate the learning rate, plateau, and difficulty level over repetitive IOS among prosthodontic residents.

MATERIAL AND METHODS

Twenty participants with no experience of IOS were recruited. Maxillary and mandibular typodont casts were attached to a manikin head. Participants received training on the IOS device (Omnicam; Dentsply Sirona) and performed maxillary, mandibular, and buccal IOS for 9 trials. The time required for scanning (ST) and the perception of difficulty were recorded. The percentage ratio of time for each trial to the best performance was calculated. A nonlinear regression model was used to estimate the learning plateau and rate.

RESULTS

The findings for maxillary, mandibular, buccal, and total IOS were as follows: ST: 106 ±40 seconds (69 ±24%), 96 ±34 seconds (68 ±22%), 74 ±31 seconds (65 ±22%), and 276 ±88 seconds (72 ±20%); learning plateau: 83.40%, 80.60%, 74.30%, and 85.10%; learning rate: 5.37, 5.01, 3.81, and 4.98. With repetition of the trials, the difficulty level decreased from 6 to 4.

CONCLUSIONS

The repetitive use of IOS showed a learning phase of 5 trials to achieve competence of 80% of their best performance. The scanning time and difficulty level decreased with the repetitive use of IOS.

Professional Variability in Decision Making in Modern Dentistry: A Pilot Study

Dental treatment planning is usually expected to take account of the individual patient's clinical risks and benefits. Ideally, the therapeutic choice for each and every patient should be based on adequate clinical diagnostics and risk assessment that facilitates stabilization of the patient's clinical condition as well as prevents further oral impairment. However, identification of the most suitable approach tends to become more and more challenging as the number of therapeutic alternatives continues to increase due to medical innovation. In this study, the challenge of decision making in modern dentistry is illustrated using the example of bounded edentulous spaces. Many therapeutic alternatives exist for such clinical scenarios, including a noninvasive monitoring approach, minimally invasive tooth recontouring, orthodontic and prosthodontic treatment, and implant placement. The findings of this pilot study highlight the utmost relevance of incorporating individual patients' needs and risks into clinical treatment planning and providing appropriate guidelines.

A progressive assessment strategy improves student learning and perceived course quality in undergraduate physiology

In 2010, second-year physiology (n = 165) had a traditional single 3-h end-of-semester exam. To provide diagnostic feedback earlier, for students enrolled in 2011 (n = 128), we incorporated an in-class exam at 3 wk in addition to the final exam. Based on initial analysis and positive student comments, for the 2012 cohort (n = 148), we expanded this to incorporate four 1-h in-class exams every 3 wk plus a short final integrative exam. Average scores from exams and questionnaires (student evaluations of learning and teaching, 10 questions) were compared among 2010, 2011, and 2012 cohorts. We also compared scores in the practical component of the course, which had a constant format for all cohorts. Data are given as means ± SD; statistical analyses were done with unpaired two-way Students t-tests. From 2010 to 2012, there was a significant improvement in total exam scores (59.7 ± 15.8 vs. 68.6 ± 14.2, P < 0.001) but no significant change in total practical scores (72.3 ± 9.0 vs. 74.4 ± 10.2, P = 0.05), indicating that the rise in exam score was not due to higher academic abilities of the 2012 cohort. Overall mean student evaluation of learning and teaching responses (4.9 ± 0.4 vs. 5.3 ± 0.3, P = 0.015) and overall percent broad agreement (66.0 ± 8.0 vs. 79.2 ± 7.5, P = 0.003) indicated a significant improvement in student satisfaction. In conclusion, both learning outcome and perceived course quality were enhanced by the increased frequency of examinations, possibly by promoting consistent student study habits.

Motivating student learning using a formative assessment journey

Providing formative assessment opportunities has been recognised as a significant benefit to student learning. The outcome of any formative assessment should be one that ultimately helps improve student learning through familiarising students with the levels of learning required, informing them about gaps in their learning and providing feedback to guide the direction of learning. This article provides an example of how formative assessments can be developed into a formative assessment journey where a number of different assessments can be offered to students during the course of a module of teaching, thus utilising a spaced-education approach. As well as incorporating the specific drivers of formative assessment, we demonstrate how approaches deemed to be stimulating, interactive and entertaining with the aim of maximising enthusiasm and engagement can be incorporated. We provide an example of a mixed approach to evaluating elements of the assessment journey that focuses student reaction, appraisal of qualitative and quantitative feedback from student questionnaires, focus group analysis and teacher observations. Whilst it is not possible to determine a quantifiable effect of the assessment journey on student learning, usage data and student feedback shows that formative assessment can achieve high engagement and positive response to different assessments. Those assessments incorporating an active learning element and a quiz-based approach appear to be particularly popular. A spaced-education format encourages a building block approach to learning that is continuous in nature rather than focussed on an intense period of study prior to summative examinations.

Introduction of an e-portfolio in clinical dentistry: staff and student views

INTRODUCTION

An electronic portfolio was introduced for undergraduate students in the School of Dental Sciences at Newcastle University. Its introduction was evaluated in terms of both staff and student response.

METHODS

A quantitative-qualitative methodology was adopted. Student views were examined quantitatively using a Likert scale based questionnaire both pre- and post-introduction of the eportfolio. Staff views were examined qualitatively by the use of focus groups.

RESULTS AND DISCUSSION

Findings included that the system was easy to use and it provided a large quantity of high quality data. The aim of the system to improve reflection and feedback was not perceived as a benefit by staff or students. The need for training was highlighted and a major disadvantage of the system was its time consuming nature. The evaluation has lead to further development of the system and continued evaluation will be important.

Invited commentary: Issues with assessing competence in undergraduate dental education

Competence of practising dentists is an issue that is being addressed in both the undergraduate and postgraduate environment in dental education. The Association of Dental Education in Europe has opened the debate further with guidelines for qualifying dentists in Europe. This puts further pressure on dental schools to produce graduates who are transparently competent. This invited commentary examines the differing concepts of competence and how they might be assessed in the undergraduate dental curriculum.

The retention of suturing skills in dental undergraduates

BACKGROUND

The ability to place sutures is an essential skill in dentistry. In our institution, the introduction of a video teaching aid and a checklist-based assessment has improved the standardisation of teaching as well as the transparency and objectivity of the assessment of suture skills. The aim of this study was to determine whether this skill once acquired is retained in the short term.

MATERIALS AND METHODS

The same cohort of 67 fourth-year students performed the same assessment at two different time points 2 months apart. The first was a formative assessment and the second was a summative assessment. Feedback was collected on both occasions.

RESULTS

On the first sitting of the assessment, 93% of the students were successful compared with 79% on the second occasion. Student feedback was similar on both occasions with 77% and 75% reporting that they thought it was an easy exam. On the first occasion, 27% thought that it was a stressful examination compared with 43% on the second occasion, which may have affected their performance.

CONCLUSIONS

These findings suggest that regular reinforcement is required to avoid deterioration of suturing skills even in the short term. This might involve exposing the students to surgery earlier in the course and introducing regular formative assessments which the students appear to associate with less stress.

A generic consensus assessment of undergraduate competence in forceps exodontia in the United Kingdom

The assessment of competence in clinical skills has become more frequent in published healthcare curricula and syllabuses recently. There are agreed mechanisms for the assessment of competence in the post-graduate environment, but no consensus within the undergraduate curriculum. This paper seeks to develop an agreed generic checklist for the assessment of competence in forceps exodontia.

MATERIALS AND METHODS

A modified Delphi process was undertaken with representatives from all UK dental schools (n = 13) to develop a generic checklist for the assessment of competence in forceps exodontia. A content analysis of the assessments employed by each school was used to help discussion and inform the Delphi process.

RESULTS

Seven schools currently employ a summative assessment of competence in forceps exodontia, with the majority employing a structured clinical objective test (n = 6). From the seven assessments, there were a total of 29 putative items and 10 putative domains identified for a generic checklist. These were reduced to five domains and 19 items through the content analysis and Delphi process, and a generic overarching checklist was created.

CONCLUSION

Using this generic checklist, it may now be possible to pool data inter-institution to perform more powerful analyses on how our students obtain, or fail to obtain competence in forceps exodontia.