Evaluating student performance assessment methods in Objective Structured Clinical Exam: perspectives and comparisons

PURPOSE

This study aims to evaluate how student performance and perspectives changed when the Objective Structured Clinical Exam (OSCE) assessment system was changed from a composite score to discipline-specific grading at the Harvard School of Dental Medicine.

METHODS

The retrospective study population consisted of all students (n = 349) who completed three OSCEs (OSCE 1, 2, and 3) as part of the predoctoral program during the years 2014-2023. Data on the students' OSCE scores were obtained from the Office of Dental Education, and data on students' race/ethnicity and gender were obtained from their admissions data.

RESULTS

The likelihood of a student failing the OSCE after the assessment system change significantly increased with an adjusted odds ratio of 20.12. After the change, the number of failed subjects per student decreased with an adjusted mean ratio of 0.48. Students perceived the OSCE as being less useful after the change. Independent of the grading change, OSCEs 1 and 2 were seen as more useful compared to OSCE 3, which is administered in the last year of the Doctor of Dental Medicine program.

CONCLUSION

The discipline-specific nature of the new assessment system helps focus on specific areas of remediation, rather than blanket remediation used previously, in order to isolate the actual areas of deficiency and to focus remediation efforts so that students can align their learning needs appropriately. Therefore, although the actual number of fails identified increased for the course, the assessment change has allowed for more directed, actionable information to be gained from the OSCE to prepare students to work toward competency standards.

Predictors of procedural errors in class II resin composite restorations using bitewing radiographs

Objective

To identify the potential factors that induce procedural errors during posterior proximal resin composite restorations placed by dental students.

Materials and Methods

This retrospective study evaluated 803 bitewing radiographs of posterior proximal resin composite restorations placed by dental students at Imam Abdulrahman bin Faisal University. Atypical radiographic signs of failure were screened, and different patient-, operator-, and clinical-related factors were recorded. Chi-square test was used to examine the relationship between procedural errors and recorded factors. Stepwise adjusted logistic regression model was performed to identify predictors of procedural errors.

Results

The most observed errors were internal gaps at the bonding interface and internal voids. Molars had 0.39 the risk of internal voids (odds ratio [OR] = 0.39; confidence interval [CI] = 0.25-0.60; P = <0.0001), 0.41 the risk of sharp angle (OR = 0.41; CI = 0.24-0.68; P = <0.001), and 0.57 the risk of open contact (OR = 0.57; CI = 0.34-0.97; P = 0.04) compared to premolars. Those who were >40 years of age had 1.79 the risk of overhang compared to younger patients (OR = 1.79; CI = 1.04-3.11; P = <0.04). First molars and premolars had 0.64 the risk of overhang compared to second molars and premolars (OR = 0.64; CI = 0.41-1.00; P = 0.04). Junior students had 1.97 the risk of internal gap compared to their senior counterparts (OR = 1.97; CI = 1.20-3.21; P = 0.008). Mesial restorations had 0.38 the risk of external gap compared to mesio-occluso-distal (MOD) restorations (OR = 0.38; CI = 0.19-0.78; P = 0.003). Restorations with a margin coronal to the cemento-enamel junction (CEJ) had 0.44 the risk of external gap compared to those restorations with a margin apical to the CEJ (OR = 0.44; CI = 0.29-0.66; P = <0.0001).

Conclusion

Our findings suggested a higher incidence of procedural errors in restoring premolars and MOD cavity preparations. Therefore, it is crucial to enhance the comprehensiveness of laboratory training and expose students to diverse clinical scenarios and various techniques.

The use of objective structured clinical examination in dental education- a narrative review

The Objective Structured Clinical Examination (OSCE) is a performance-based assessment intended to assess medical students' clinical competency in a simulated, standardized environment. Because it measures the student's ability to use clinical knowledge, diagnostic skill, and decision-making, the OSCE is thought to be more objective than traditional tests. OSCE exams have been increasingly employed in dentistry schools, particularly in the last decade, and it is crucial to investigate instructors' and dental students' experiences with this evaluation approach.

Virtual and Interprofessional Objective Structured Clinical Examination in Dentistry and Dental Technology: Development and User Evaluations

BACKGROUND

Interprofessional education (IPE) facilitates interprofessional collaborative practice (IPCP) to encourage teamwork among dental care professionals and is increasingly becoming a part of training programs for dental and dental technology students. However, the focus of previous IPE and IPCP studies has largely been on subjective student and instructor perceptions without including objective assessments of collaborative practice as an outcome measure.

OBJECTIVE

The purposes of this study were to develop the framework for a novel virtual and interprofessional objective structured clinical examination (viOSCE) applicable to dental and dental technology students, to assess the effectiveness of the framework as a tool for measuring the outcomes of IPE, and to promote IPCP among dental and dental technology students.

METHODS

The framework of the proposed novel viOSCE was developed using the modified Delphi method and then piloted. The lead researcher and a group of experts determined the content and scoring system. Subjective data were collected using the Readiness for Interprofessional Learning Scale and a self-made scale, and objective data were collected using examiner ratings. Data were analyzed using nonparametric tests.

RESULTS

We successfully developed a viOSCE framework applicable to dental and dental technology students. Of 50 students, 32 (64%) participated in the pilot study and completed the questionnaires. On the basis of the Readiness for Interprofessional Learning Scale, the subjective evaluation indicated that teamwork skills were improved, and the only statistically significant difference in participant motivation between the 2 professional groups was in the mutual evaluation scale (P=.004). For the viOSCE evaluation scale, the difference between the professional groups in removable prosthodontics was statistically significant, and a trend for negative correlation between subjective and objective scores was noted, but it was not statistically significant.

CONCLUSIONS

The results confirm that viOSCE can be used as an objective evaluation tool to assess the outcomes of IPE and IPCP. This study also revealed an interesting relationship between mutual evaluation and IPCP results, further demonstrating that the IPE and IPCP results urgently need to be supplemented with objective evaluation tools. Therefore, the implementation of viOSCE as part of a large and more complete objective structured clinical examination to test the ability of students to meet undergraduate graduation requirements will be the focus of our future studies.

Adaptation to Virtual Assessment during the COVID-19 Pandemic: Clinical Case Presentation Examination

BACKGROUND

Case presentation assessment is common in both medicine and dentistry and is known under various names depending on the country and institution. It relates mainly to aspects of diagnosis and treatment planning and is considered highly authentic and useful. The COVID-19 pandemic necessitated the movement of this assessment from face-to-face to online. The aim of this investigation was to explore the students' impressions of the two different examination modalities. With this information, a decision on future diets of this examination can be made to accommodate the students' perspectives.

METHODS

Quantitative and qualitative data were gathered using an online, self-administered survey.

RESULTS

The students were split 50/50 regarding which assessment modality they preferred. Overall, they considered the online examination to be fair, and the majority agreed that the online format allowed them to display their knowledge as well as face-to-face.

CONCLUSIONS

The delivery of case presentation examination is possible online. An online case presentation is a fair, useful, and authentic assessment that is appropriate to the needs of the faculty and students. Satisfaction with the two possible methods of conducting this assessment suggests it would be reasonable to conduct this examination online in the future.

The Objective Structured Clinical Examination (OSCE) in Periodontology with Simulated Patient: The Most Realistic Approach to Clinical Practice in Dentistry

The objective structured clinical examination (OSCE) is becoming an increasingly established assessment test in dental schools. The use of simulated patients in the OSCE makes the stations more similar to clinical practice. Therefore, the student can show their technical and clinical knowledge, and certainly, their ability to manage the patient. These sorts of tests, in which simulated patients can be included, would be used before the student started clinical practice with patients and/or at the end of the degree. The objective of this work was to describe how the periodontology station was developed using a simulated patient for students of a fifth year dentistry degree taking an OSCE test. Furthermore, a questionnaire was created to learn the perception of the students about this station and its characteristics. The fifth year students at the European University of Madrid positively evaluated this station in their examination. In addition, it was recorded that they preferred a simulated patient in their tests, rather than stations with clinical cases, images, X-rays, and presentations. It is essential that once the OSCE has been completed, the student receives a feedback to learn where they have failed and, therefore, be able to improve any of the aspects evaluated in the station.

The multi-factor analysis of computer-based test scores in objective structured clinical examination of clerkship in school of dentistry, National Taiwan University

Background/purpose

The objective structured clinical examination (OSCE) is a test designed to test technical, clinical, and practical competency in medicine teaching. This study tried to evaluate the influencing factors of the computer-based tests (CBT) results for dental students.

Materials and methods

The 32 5th-year dental students of the dental school of National Taiwan University (NTU) joined preclinical CBT. The contents of CBT included clinical knowledge and practical skills of nine dental specialties. Twenty single-choice questions were played with the PowerPoint 2016 on a desktop, and the playing time for each question was 30 s. Several factors including the gender, age, and dental student admission ways (with or without the skill tests) were evaluated. Also, the comparison between the clerk's grade and CBT scores of nine dental specialties was performed. The students also filled the feedback questionnaire about the examination questions and suggestions for the CBT.

Results

There were no significant differences in the mean CBT scores between male and female dental students, between two groups of dental students with or without the skill tests in the admission process, and among three different age groups of dental students. The clerk's scores were not directly related to the CBT scores.

Conclusion

The CBT has been well-intentioned in dental education since 2022 in Taiwan. The CBT results revealed that the conjunction of knowledge from the textbook and clinical observation should be enhanced. The guideline should be established to integrate CBT into preclinical education and promote the performance of clinical work of dental students.

Dental students' perceptions of the long case

INTRODUCTION

The long case examination is used to assess clinical competency in dental education. However, the academic literature, much of which is in medical education, highlights concerns regarding the relevancy and authenticity of the long case. To date, dental students' experiences of the long case have been under-researched. This study examines students' experiences and perceptions of the long case examination at an Australasian dental school.

MATERIALS AND METHODS

This study was a qualitative investigation. Students participated in interviews to discuss their perceptions and experiences of the long case examination. The interviews were voice-recorded and transcribed, and a thematic inductive analysis was undertaken.

RESULTS

Three main themes emerged from the data: stress, where students described stressors before, during and after the long case; fairness, where students positioned the long case as either fair or not fair; and confusion, where students spoke about their perceived lack of understanding of the examination process and procedures.

CONCLUSION

The concerns students raised regarding stress, fairness and confusion are considered and ways in which the long case might be developed in order to support students' learning are presented. Alternative structures or practices that might be explored include greater calibration of examiners and cases, and enhancements to how students are prepared for and prepare for the examination. The results of this research will inform ongoing development of assessment practices.

Admissions data as predictors for student performance in restorative dentistry

INTRODUCTION

The purpose of the study was to determine the relative predictive power of admissions criteria and predoctoral academic performance in the area of restorative dentistry in order to better understand and guide the dental school admissions process.

METHODS

In addition to admissions criteria, student performance in restorative dentistry, such as the Final Restorative Treatment course, the didactic and preclinical examinations, and the objective structured clinical examinations (OSCE), was measured for the 172 predoctoral students representing the graduating classes of 2017-2021.

RESULTS

The association between the didactic score in restorative dentistry and grade point average (GPA, p = 0.009) and Dental Admission Test (DAT, p = 0.002) score was statistically significant. Although no statistically significant association was found between preclinical scores and gender, GPA, or DAT, there was a statistically significant association between preclinical scores and the Perceptual Ability Test (PAT) score (p = 0.012). Preclinical scores included laboratory examinations for fixed prosthodontics, such as crown preparation and temporization, representing hand skills assessment. Finally, there was no statistically significant association found between the OSCE score and any of the admissions criteria.

CONCLUSION

The dental admissions criteria could play a role in predicting academic performance in the didactic portion and preclinical component of restorative dentistry.

Reflective writing in dental education to improve critical thinking and learning: A systematic review

The Commission on Dental Accreditations states that "graduates must be competent in the use of critical thinking and problem-solving." With this in mind, dental education programs continually strive to enhance and deepen these skills by incorporating effective instructional strategies into the curriculum. To do this, predoctoral dental education has used a variety of techniques including problem-based learning, OSCEs, and standardized patient activities. Another technique, while less popular but potentially more effective if done well, is reflective writing. This study aimed to assess the effect of reflective writing on students' critical thinking skills and learning in dental education programs. Two database searches conducted between 2000 and 2019 resulted in 317 articles after the initial screening. Ultimately, 13 articles met inclusion criteria. The following recurring factors were identified for inclusion in dental education reflective writing initiative: reflective writing interventions, reflective writing instructions, writing prompts, evaluation frameworks, and negative perceptions of reflective writing. In answering the key aim of this review, 12 of the 13 studies determined that reflective writing had a positive impact on students' critical thinking, judgment, and/or learning. Key findings included positive effects on student growth in reflection, learning through reflective writing, reflection skills, self-assessment, critical thinking, clinical reasoning, problem solving, and motivation to change after evaluated experiences. The review indicated that dental education programs, which implemented reflective writing as an assessment tool within the curriculum as a means of developing and deepening critical thinking skills and learning were by and large successful in this effort.

Early Clinical Exposure in U.S. Dental Schools and Correlation with Earlier Competencies Evaluation

Early clinical exposure (ECE), defined as any interaction with patients prior to the portion of the curriculum when den- tal students spend most of their time at school as a primary provider, is a growing trend in curriculum reform across U.S. dental schools in the 21st century. The aims of this study were to characterize the types of ECE implementation in U.S. dental schools and determine if ECE correlated with earlier clinical competency assessments. In September 2018, the academic deans of all 66 U.S. dental schools were invited to respond to an eight-item electronic survey about ECE at their schools. Representatives of 40 schools submitted complete responses, for a response rate of 60.6%. Among the respondents, 85% reported their schools started their principal clinical experience (PCE), the portion of the curriculum when students spend most of their time as the primary provider for patients, during the last quarter of Year 2 or the first quarter of Year 3. Respondents at all 40 schools reported offering some form of ECE as part of the formal curriculum, with shadowing and performing dental prophylaxis the most commonly of- fered types. No statistically significant associations were found between specific types of ECE and related Commission on Dental Accreditation (CODA) clinical standards for both formative and summative assessments. Although U.S. dental schools have been incorporating more ECE into their curricula over the past decade, these findings suggest that it has not led to earlier clinical competency assessments.

Summative objective structured clinical examination as a reference of learners' need before entering their internship

Background/purpose

Objective structured clinical examination (OSCE) has found increasingly wide use in the education of clinical dentistry in recent years. The objectives of this study were to find the predictability of OSCE in representing a student's actual ability in a clinical environment and to provide instructors with better ideas to guide an individual student in a certain field.

Materials and methods

Thirty-one dentistry students' qualification OSCE scores were obtained after the conclusion of their dental clerkship in December 2014. The OSCE scores were compared with the various scores given by instructors from different departments during their internship a year later. The correlation was investigated between these two scores.

Results

The scores of the first station of OSCE-explaining a treatment plan for the restoration of a missing tooth-were correlated with the scores of prosthodontics in the internship. The scores of the fourth station of OSCE-explaining to a mother regarding the obtainment of a radiograph of her child's tooth-were correlated with the scores of orthodontics. In addition, the total score of the six OSCE stations was correlated with the scores of orthodontics, oral surgery, and pediatric dentistry during dental internship.

Conclusion

The results of clerkship qualification OSCE could provide instructors with insights into the needs of the students before their entering internship.

Objective structured clinical examination, OSCEs: an advance in the teaching and learning process in the student’s perception

Abstract Introduction The OSCE is a method of clinical competencies evaluation that has gained international popularity in medical and dental education. Objective The purpose of the present study was to describe the stages of development of the OSCE for the undergraduate course in Dentistry and to report the students' perception about this method of evaluation, regarding the degree of difficulty, time for each stage, importance of each station, number of stations, organization of the exam, as well as the total time for the OSCE. Material and method This research was an observational and cross-sectional study, composed of the carry out of an OSCE and later application of an evaluative questionnaire to the students who were in the fourth semester of the Dentistry course. These students had basic knowledge and an intermediate level of competences, compatible with their stage of education. Result As regards the number of stations, 43(97.7%) of the students responded that this was appropriate, OSCE process as a whole was very well organized (n=25; 56.8%), organized (n=17; 38.6%) or not very organized (n=1; 2.27%) and about the total time of the OSCE process, 29(65.9%) reported that it was appropriate; 10 (2.27%) said that it was short; 4 (9.09%), that it was long. Conclusion The student’s perception was positive especially regarding to organization and the time attributed to each station. Furthermore, the students considered that the topics and questions applied in each station were relevant.

Assessing Dental Hygiene Students' Test Anxiety in Examiner-Administered vs. Camera-Supervised OSCEs: A Pilot Study

The aim of this pilot study was to measure and compare dental hygiene students' level of text anxiety and performance when being tested on an objective structured clinical examination (OSCE) using camera supervision versus direct examiner observation. This randomized, controlled trial, conducted in October 2016, assessed students' anxiety levels and performance using the Zung Self-Rating Anxiety Scale and performance on an OSCE. All 32 first-year students in one U.S. dental hygiene program were invited to participate, and all 32 participated for a 100% response rate. Participants were randomly assigned to the direct observation (N=16) and camera-supervised (N=16) groups. The comparison of pre-OSCE anxiety levels and performance scores evaluated with direct observation (36.4 and 20.4) versus camera-supervised (37.5 and 20.9) found no statistically significant difference between the two groups (p=0.496, p=0.403). The difference in post-OSCE anxiety levels between the direct observation and camera-supervised groups (33.8 and 37.8) approached significance (p=0.051). This study found that student anxiety levels were similar between the two groups except for the post-OSCE direct observation anxiety levels, which were lower. No significant difference was found between the two groups' performance scores. Due to similar performance scores and anxiety levels for the direct examiner and camera-supervised methods, the camera-supervised OSCE warrants further exploration as an alternative approach to direct observation for evaluation of students' performance.

Periodontal education and assessment in the undergraduate dental curriculum-A questionnaire-based survey in European countries

OBJECTIVES

This survey aimed to evaluate whether periodontal education and assessment in undergraduate dental curricula amongst the member countries of the European Federation of Periodontology (EFP) follow the competency-based curricular guidelines and recommendations developed by the Association for Dental Education in Europe.

MATERIALS AND METHODS

A multiple-choice questionnaire was emailed to 244 dental institutes amongst the 24 EFP member countries between November 2014 and July 2015.

RESULTS

Data were received from 16 (66.7%) EFP member countries. Out of 117 responding dental institutes, 76 (64.95%) were included as valid responders. In most of the institutes (86.3%), a minimum set of competencies in periodontology was taken into account when constructing their dental education programmes. Out of 76 responders, 98.1% included lecture-based, 74.1% case-based and 57.1% problem-based teaching in their periodontal curricula, whilst a minority (15.9%) also used other methods. A similar pattern was also seen in the time allocation for these four educational methods, that is, the highest proportion (51.8%) was dedicated to lecture-based teaching and only a small proportion (5.7%) to other methods. Periodontal competencies and skills were most frequently assessed by clinical grading on clinic, multiple-choice examination (written examination) and oral examination, whereas competency tests and self-assessment were rarely used. Only in 11 (14.5%) cases, access flap procedures were performed by students.

CONCLUSION

Great diversity in teaching methodology amongst the surveyed schools was demonstrated, and thus, to harmonise undergraduate periodontal education and assessment across Europe, a minimum set of recommendations could be developed and disseminated by the EFP.