An explanation of workplace-based assessments in postgraduate dental training and a review of the current literature

Using live stream technology to conduct workplace observation assessment of trainee dental nurses: an evaluation of effectiveness and user experience

AIM/OBJECTIVE

This study evaluates the effectiveness and users' experience of using live stream technology to conduct workplace observation assessments of trainee dental nurses. Information on the usability, accessibility, and general satisfaction of this technological technique were collected.

MATERIALS AND METHODS

This nationwide cross-sectional survey was conducted in Scotland and included one focus group and three online questionnaires with qualitative and quantitative questions. The quantitative responses were described using standard descriptive analysis, while the quantitative data were investigated using thematic analysis.

RESULTS

Eighty-one trainee dental nurses, 35 clinicians and 19 assessors participated in this study. Live stream observation was generally well received by the trainee dental nurses and clinicians, who thought that it had helped increase their confidence to perform practical skills. The assessors also stated that overall satisfaction was high, and that live stream observation met their expectations for efficacy. However, several technical challenges, such as network issues were brought up by responders.

CONCLUSION

This study provides evidence that workplace observation assessments can be performed in the future by using live stream technology. However, additional investigation and comparison will aid in determining the most effective way of using this approach and providing feedback to promote learning among dental trainees.

The Competency-based Evaluation of Educational Crew of Dental Faculty’s Obstacles in Institutionalizing Performance Assessments

Aims:

This study evaluates the educational crew of dental faculty’s lived experiences facing obstacles and requirements in institutionalizing performance assessments to implement a professional competency-based evaluation system.

Background:

The competency-based evaluation of learning and teaching processes has been adopted as a key policy in the developed world, which indicates the achievement rate of educational goals and the quality of education.

Objective:

The main objective of this study was to evaluate obstacles in institutionalizing performance assessments for the educational crew.

Methods:

This qualitative study used a semi-structured interview in a focus group discussion. The experience of the educational crew regarding the obstacles of using performance assessments and their approaches to conducting a professional competency-based evaluation was assessed. The recruited participants were educational supervisors, professors of orthodontics and prosthodontics, and the medical education department and evaluation committee members of the faculty of dentistry at the University of Tabriz. The purposive sampling technique was used and continued until reaching saturation. Five focus group discussions were conducted with fourteen educational crew and three medical education department members. The data were analysed using thematic analysis.

Results:

The interview analysis results yielded 450 codes in three general categories, including “current condition of clinical education,” “obstacles of implementing new evaluation methods,” and “requirements for effective evaluation of clinical skills.” According to the results, changes in evaluation methods are necessary to respond to community needs. There are also many cultural problems with applying western models in developing countries.

Conclusion:

The medical community should be directed towards a competency-based curriculum, especially in procedure-based fields, such as dentistry.

Other:

They are moving towards altering traditional evaluation methods (the traditional classroom-based lectures). This paradigm change requires support from the department and the provision of infrastructure.

Mind the 'GAPP': a pre-graduation assessment of preparedness for practice amid a pandemic

Introduction 'Preparedness for practice' refers to a multifaceted concept, encompassing not only clinical skills, but also broader, non-clinical skills, such as communication and professionalism. Previous graduates have reported feeling less prepared for complex procedures, such as molar endodontics and surgical extractions. Dental students typically utilise their final year to refine their clinical skills, however, the COVID-19 pandemic has meant that newly qualified dentists will have had around six months where they have not performed clinical dentistry before beginning dental foundation training (DFT).

Aims This study aimed to explore final year students' self-reported preparedness for practice, identify areas of relative weakness that may influence future training needs and to highlight any perceived impact of the pandemic on final year experience and potential impact on DFT.

Materials and methods The current Graduate Assessment of Preparedness for Practice (GAPP) questionnaire was adapted for our specific research aims and piloted and the PreGAPP questionnaire distributed via the social media channels of dental school student societies. Analysis was carried out using IBM Statistical Package for the Social Sciences software using descriptive statistics and the Mann-Whitney U test for two unrelated variables.

Results Responses were received from final year students across all 16 UK dental schools. Students reported increased preparedness across domains in which they had the most experience; for example, providing preventative advice and administering local anaesthesia. Male students reported feeling significantly more prepared than female students, mature students significantly more than younger students, and students on four-year courses significantly more than traditional five-year courses. The COVID-19 pandemic was expected to have a major-to-severe impact on undergraduate experience and future DFT prospects.

Conclusion Dental graduates in the COVID-19 era may have significantly different training needs to those before them. Complex clinical procedures remain the areas where students feel they are least prepared for practice. The importance of a clinical passport to highlight current experience level to trainers, alongside the creation of a personal development plan at the beginning of DFT, will ensure that targeted and personalised training can be implemented where required.

Illustrates the perceived effect of the COVID-19 pandemic on final year students' preparedness for practice and the impact it may have on dental foundation training through quantitative and qualitative methods.

Highlights and explores the differences in the perceptions of preparedness for practice of final year students across sex, age and course length.

Explores various aspects of clinical supervision and assessment and reinforces the importance of appropriate student-staff ratios to ensure clinical learning is effective.

Implementation and Use of Workplace-Based Assessment in Clinical Learning Environments: A Scoping Review

PURPOSE

Workplace-based assessment (WBA) serves a critical role in supporting competency-based medical education (CBME) by providing assessment data to inform competency decisions and support learning. Many WBA systems have been developed, but little is known about how to effectively implement WBA. Filling this gap is important for creating suitable and beneficial assessment processes that support large-scale use of CBME. As a step toward filling this gap, the authors describe what is known about WBA implementation and use to identify knowledge gaps and future directions.

METHOD

The authors used Arksey and O'Malley's 6-stage scoping review framework to conduct the review, including: (1) identifying the research question; (2) identifying relevant studies; (3) study selection; (4) charting the data; (5) collating, summarizing, and reporting the results; and (6) consulting with relevant stakeholders.

RESULTS

In 2019-2020, the authors searched and screened 726 papers for eligibility using defined inclusion and exclusion criteria. One hundred sixty-three met inclusion criteria. The authors identified 5 themes in their analysis: (1) Many WBA tools and programs have been implemented, and barriers are common across fields and specialties; (2) Theoretical perspectives emphasize the need for data-driven implementation strategies; (3) User perceptions of WBA vary and are often dependent on implementation factors; (4) Technology solutions could provide useful tools to support WBA; and (5) Many areas of future research and innovation remain.

CONCLUSIONS

Knowledge of WBA as an implemented practice to support CBME remains constrained. To remove these constraints, future research should aim to generate generalizable knowledge on WBA implementation and use, address implementation factors, and investigate remaining knowledge gaps.

A qualitative study of trainer and trainee perceptions and experiences of clinical assessment in post-graduate dental training

BACKGROUND

The implementation of workplace-based assessment (WBA) needs to ensure the achievement of pre-set competences but may look different across varying contexts, such as in post-graduate dental education. The purpose of this study is to explore the perception of residents, faculty members and alumni concerning their experience with clinical assessment, and what configurations they consider as optimal to maximise the entailed learning experience.

METHODS

This study relied on a qualitative descriptive design using two data collection tools: focus group sessions, and semi-structured, one-to-one interviews. Data were triangulated from three sources: residents, faculty members and alumni. The data were inductively analysed based on constructivist epistemology. This was done using the Thematic Analysis approach, facilitated by NVivo software.

RESULTS

The analysis revealed two mutually exclusive themes: process and people. Within process, variables related to quality, workflow and feedback surfaced. As for the people theme, the main two group of stakeholders referred to in the related analysis were the trainees and the trainers.

DISCUSSION

There are many variables that need to be considered when developing an evidence-driven WBA. In addition, factoring into the design of the WBA the perception of the main stakeholders will enable contextualisation which is expected to raise the reliability of the adapted tools.

CONCLUSION

This study introduced a framework that could support post-graduate universities in their journey towards developing context-specific WBA.

Adaptation, psychometric properties, and implementation of the Mini-CEX in dental clerkship

BACKGROUND

Workplace-based assessment is a key component of dental-student clerkships, allowing students to demonstrate clinical proficiency.

PURPOSE

This study adapts the Mini-Clinical Evaluation Exercise (Mini-CEX) to a dentistry-program clerkship, analyzing the results and examining the psychometric properties of Mini-CEX.

METHODS

First, Delphi panel methodology was used to ensure content validity. Mini-CEX was then piloted in the dental-clerkship program, with each student assessed by at least 2 supervisors and a peer student. Subsequently, psychometric properties, acceptability, and observation time were analyzed.

RESULTS

The study was conducted between July and November 2019. Overall, 140 Mini-CEX evaluation exercises were carried out on 30 students by 84 supervisors and 56 peers. The adapted instrument was found to be unidimensional, obtaining an acceptable internal consistency (α = 0.74). As the assessor type changed, there were differences in observation time; the medians (Q1-Q3) were 10 minutes (5-15) for supervisors and 30 minutes (20-45) for peer students (P < 0.001). This difference was also observed in assessor perceptions (P < 0.001), with supervisors scoring a median of 6 (6-6.75) and peer students scoring a median of 7 (6-7). No differences were found between supervisor and peer scores.

CONCLUSION

The adapted version of Mini-CEX can objectively assess the clinical performance of dental students, achieving validity and reliability values similar to those obtained in the original instrument.

Effect of individual structured and qualified feedback on improving clinical performance of dental students in clinical courses-randomised controlled study

AIM

Analysis of the effect of individual structured and qualified feedback (FB) on practical skills development of dental students during clinical courses.

METHODS

Fifty-three final-year dental students at Jena University Hospital participated in this prospective randomised controlled interventional study. Two calibrated assessors evaluated 128 pre- and post-assessments of 4 different dental treatment steps performed by dental students during the integrated clinical course in restorative dentistry and prosthodontics and the clinical course paediatric dentistry. The assessment included direct observation, graded and non-grading evaluation and was documented with a specific FB assessment tool. Dental students in the intervention group (IG) received an elaborated, structured and qualified FB after the pre-assessment that focussed on individual strengths and weaknesses, providing specific suggestions for improvement and establishing a personal learning goal. Participants were randomly allocated to the IG and the control group (CG).

RESULTS

In both groups, dental students significantly enhanced their performance, but the difference was higher in the IG than in the CG. Large effect sizes (ES) were observed in all observed items, but FB had largest effect size in improving technical skills (ES = 1.6), followed by management (ES = 1.3) and communication skills (ES = 0.8). Factors with the highest influence on FB in enhancing dental students' clinical performance were their insight into their own mistakes or omissions, the observed dental treatment step and the duration of FB.

CONCLUSION

Individual structured and qualified FB is an effective method to enhance dental students' professional performances and to individually guide the learning process.

Sawbones laboratory in orthopedic surgical training

Sawbones are artificial bones designed to simulate the bone architecture, as well as the bone’s physical properties. The incorporation of sawbones simulation laboratories in many orthopedic training programs has provided the residents with flexibility in learning and scheduling that align with their working hour limitations. This review paper deliberates the organization of sawbones simulation in orthopedic surgical training to enhance trainee’s future learning. In addition, it explores the implications of sawbones simulation in orthopedic surgical teaching and evaluation. It scrutinizes the suitability of practicing on sawbones at the simulation laboratory to improve orthopedic trainee’s learning. This will be followed with recommendations for future enhancement of sawbones simulation-based learning in orthopedic surgical training.

Utilisation and outcomes of case-based discussion in otolaryngology training

OBJECTIVES

To assess the reliability, validity and outcomes of Case-based Discussion (CBD) in otolaryngology training.

DESIGN

Retrospective database analysis.

SETTING

National electronic database.

PARTICIPANTS

North London otolaryngology trainees.

MAIN OUTCOME MEASURES

We tested the tool's reliability along with its capacity to denote trainee progress. A score was calculated (cS) and compared across core (CT) and specialty trainees (ST) at all levels. The number of items rated as "development required" (D) was also examined.

RESULTS

One thousand four hundred and fifty-six CBDs were submitted by 46 trainees from 2007 to 2013, averaging 13.6 per trainee per year. Items relating to knowledge, management and judgement were more popular (98% usage), and better predictors of cS compared to other parameters (rs: +0.74, +0.70 and +0.72, respectively). CBD was found to be reliable (Cronbach's α = 0.848) and highly sensitive (99%), yet not specific. cS was significantly higher in ST than CT (95.3% ± 0.6 versus 88.7% ± 1.3). pS showed a similar pattern (3.15 ± 0.4 versus 2.0 ± 0.05) (P < 0.001). cS and pS increased from CT1 to ST8 (rs: +0.60 and +0.34, respectively). The number of D-rated items decreased with increasing year of training.

CONCLUSION

Case-based discussion is a reliable and valid tool in otolaryngology training. It is highly sensitive but not specific. Trainees should be encouraged to use it at all levels.