The development of an assessment system for dental vocational training and general professional training: a Scottish approach

Is the number of procedures completed a valid indicator of final year student competency in operative dentistry?

Introduction Concerns regarding the preparedness of dental graduates often focus on the number of procedures undertaken before qualification.

Aims To assess the validity of numerical requirements in determining whether a student has sufficient competency to graduate.

Materials and methods In total, 50,821 observations of student activity while on the restorative clinic, for two integrated student cohorts (n = 139), were continuously recorded using the LiftUpp system from their third year to graduation. Analysis of data determined the most frequent type, location and difficulty of the direct restorations provided.

Results Four groups of students, categorised by the total number of restorations they provided, were identified. Those with the most restorations (70+) had a statistically significantly greater experience in restoring different tooth surfaces, in different quadrants, on different patients than those with the least (40-49). However, increased numbers did not correlate with increasing difficulty, suggesting that students simply became more adept at easier procedures.

Discussion Using a derived measure of 'consistency', it was observed that increased performance consistency was not correlated with greater overall experience. Furthermore, only 4% of students were 100% consistent in their performance in their final year.

Conclusions These data suggest that a 'numerical requirements' approach cannot be used, in isolation, as a valid measure of the competency of an individual. Further work is needed to validate consistency as a holistic measurement to characterise the performance of graduates.

The numerical requirement approach alone should not be used to make valid judgements over the competency of a dental student to undertake direct restorations.

Consistency as a holistic measure of performance may be more suitable to assess the set of skills required.

Students are not usually 100% consistent in their performance by graduation.

The risks and benefits of social media in dental foundation training

The use of social media has greatly expanded in the last decade, with widespread use of smartphones, the internet, and other multimedia to enhance learning. There is evidence to suggest that social media has a place in healthcare education, but there is limited research to suggest the effectiveness or use of it in dental foundation training. This paper discusses the risks and benefits of social media and suggests that a better understanding of social media and its role in the development and practice of newly qualified dental professionals could benefit both trainees and trainers.

The undergraduate preparation of dentists: Confidence levels of final year dental students at the School of Dentistry in Cardiff

Objective To investigate the self-reported confidence and preparedness of final year undergraduate students in undertaking a range of clinical procedures.Methods A questionnaire was distributed to final year dental students at Cardiff University, six months prior to graduation. Respondents rated their confidence in undertaking 39 clinical procedures using a 5-point scale (1 = can undertake on own with confidence, 5 = unable to undertake). Students also responded yes/no to experiencing four difficulties and to three statements about general preparedness.Results 71% (N = 51) responded of which 55% (N = 28) were female. Over half reported being 'anxious that the supervisor was not helping enough' (57%) and 'relying heavily on supervisor for help' (53%). Eighty percent 'felt unprepared for the clinical work presented' and gender differences were most notable here (male: 65% N = 33; females: 93% N = 47). Mean confidence scores were calculated for each clinical procedure (1 = lowest; 5 = highest). Confidence was highest in performing 'simple scale' and 'fissure sealant' (mean-score = 5). Lowest scores were reported for 'surgical extractions involving a flap (mean-score = 2.28)', 'simple surgical procedures' (mean-score = 2.58) and the 'design/fit/adjustment of orthodontic appliances' (mean-score = 2.88).Conclusions As expected complex procedures that were least practised scored the lowest in overall mean confidence. Gender differences were noted in self-reported confidence for carrying out treatment unsupervised and feeling unprepared for clinical work.

Programmatic assessment: From assessment of learning to assessment for learning

In assessment a considerable shift in thinking has occurred from assessment of learning to assessment for learning. This has important implications for the conceptual framework from which to approach the issue of assessment, but also with respect to the research agenda. The main conceptual changes pertain to programmes of assessment. This has led to a broadened perspective on the types of construct assessment tries to capture, the way information from various sources is collected and collated, the role of human judgement and the variety of psychometric methods to determine the quality of the assessment. Research into the quality of assessment programmes, how assessment influences learning and teaching, new psychometric models and the role of human judgement is much needed.

What influences intention to become a postgraduate trainer?

OBJECTIVE

Only a small proportion of eligible dentists become dental vocational trainers. The aim of this study was to apply Social Cognitive Theory to further an understanding of beliefs underpinning intention to become a dental vocational trainer.

SUBJECTS

Primary care dentists.

DESIGN

A self-administered questionnaire was distributed at all continuing professional development courses during November 2004 regardless of course enrolment numbers, content or duration.

OUTCOME MEASURE

Intention to train. Predictive measures Attitude toward being a trainer, attitude toward attending the trainer course, and confidence in ability to train (training self-efficacy).

RESULTS

Of the 316 respondents, 47% intended never to train. Training self-efficacy accounted for the majority of variance in intention to train (15%, step 1) with attitude to training accounting for a further 4% (step 2) (Hosmer and Lemeshow test: Chi-square = 7.36, df = 8, p = 0.498; 95% CI for EXP(b): training self-efficacy = 1.38-2.06, attitude = 1.17-2.08). Attitude to the trainer course did not enter the regression model. The most influential belief relating to intention to train was confidence in teaching, and for intention never to train was evaluating a trainee's skills. Both intenders and non-intenders believed training and attending training courses would be time-consuming and financially costly, suggesting these are not primary factors in making the decision to become a trainer.

CONCLUSIONS

Using a psychological model provided a starting point for understanding intention to become a vocational trainer and a rationale for the format of future interventions. Results suggest intention to train may be influenced by targeting dentists' confidence in teaching and performing evaluative aspects of training.

Assessment in dental vocational training: where are we now?

In this paper the author gives his opinion on the existing arrangements for assessment of vocational dental practitioners (VDPs) (outside Scotland), during dental vocational training. He critically evaluates the arrangements in relation to the aims and objectives for dental vocational training. Potential improvements to the existing system are then proposed and new assessment techniques suggested that may help to provide a more robust and relevant educational assessment strategy for dental vocational training in the future.

Key skills for newly qualified dentists: an evaluation of a West Midlands initiative

INTRODUCTION

This paper reports an evaluation of the West Midlands Key Skills initiative that provides a framework for learning during vocational training (VT).

METHOD

The 48 vocational dental practitioners (VDPs) who began their VT in August 2001 in the West Midlands were surveyed at the start and end of training (45 completed both surveys). They rated their confidence and experience in the 31 components of the Six Key Skills on visual analogue scales. Views were elicited in a survey of both VDPs (47 returns) and their general practice trainers (44 returns). Semi-structured interviews were also held with a stratified sample of 9 trainers and all four VT advisors.

RESULTS

In terms of VDP progress, a statistically significant increase in confidence and experience was found in each of the 31 components. Six themes were identified in the views data. (i) Supporting the development of Key Skills; (ii) workload implications; (iii) the "right" six? (iv) links with the advanced diploma (MFGDP(UK)); (v) assessment of VT; and, (vi) consistency and quality assurance.

CONCLUSION

VT successfully develops the confidence and experience of newly qualified dentists in the Six Key Skills and has been well received by the majority of VDPs, trainers and advisors in the West Midlands.

'On Track' – an educational resource to support dental SHO training

This paper describes the development, implementation and evaluation of an educational resource, "On Track", designed to support dental SHOs throughout their time in post, thereby helping them gain optimum benefit from this important stage in their early postgraduate training. The need for such a resource was highlighted by a previous study undertaken by the paper's key authors with SHOs and their trainers in a UK dental school. Among several areas of change identified by both trainers and trainees were the needs to develop the SHO experience into more than an apprenticeship by improving the trainer/trainee relationship, and to encourage trainee-centred learning and self-appraisal. The key aims of "On Track" were therefore to define the outcomes of SHO training, to facilitate the relationship between SHO and educational supervisor (trainer), and to encourage the SHO in reflective learning and self-appraisal."On Track" was subject to developmental testing with both trainers and trainees to ensure that it could achieve these aims and the resultant resource is described, along with its implementation and evaluation. "On Track" met with mixed success and the reasons behind this and conclusions drawn for the introduction of any future similar resources are outlined.

The specification of learning outcomes in dentistry

Increasing emphasis is being placed in dentistry, as in other areas, on outcome-based education and on the specification of learning outcomes. This paper describes the adaptation for dentistry of the medical three circle model as described by Harden to specify learning outcomes. The model offers an effective and user-friendly format based on the three dimensions of the work of a dentist. What the dentist is able to do ('doing the right thing'), how the dentist approaches their practice ('doing the thing right'), and the dentist as a professional ('the right person doing it'). The model also facilitates curriculum and examination planning, reconciles tensions between vocational and academic education and is readily understood and accepted by students and teachers.

Satisfactory completion of dental vocational training in Scotland: a system of assessment

Formal assessment of dental vocational training (DVT) has been an issue since 1993 when DVT became mandatory for graduates wishing to practice within the NHS. There have been a number of other drivers for change, including concerns about the capabilities of new graduates, a lack of standardisation of the training experience and accountability for standards of training. All of these factors contributed to the decision, by the Scottish Council for Postgraduate Medical and Dental Education (now part of NHS Education for Scotland--NES), to support a programme of research into the development of a competency-based system of assessment for DVT. During the life of this project the argument for introducing assessment of DVT and General Professional Training (GPT) has become ever stronger.

Comprehensive validation of competencies for dental vocational training and general professional training

This paper outlines a study designed to validate competencies for dental vocational training (DVT) and general professional training (GPT) in order to ensure their accuracy and acceptability. A highly inclusive approach is described whereby all trainers in Scotland were invited to participate in the exercise. The 168 individuals recruited were drawn from all branches of the dental services and all regions in Scotland. Using online or paper questionnaires, quantitative and qualitative data were collected for each competency statement over 9 months, after which focus groups discussed and decided which changes should be made. A high response rate was observed and from the 160 competencies originally identified, almost half (47.5%) were redrafted as a direct result of the validation process. Sections of the competency document that required most attention are discussed, as are the nature of changes made to the competencies. As a result of this study, a fully validated competency document for DVT and GPT has been produced and will allow a high degree of standardization of training through the provision of essential consistent information to trainers and VDPs.

Facing the challenges of competency-based assessment of postgraduate dental training: Longitudinal Evaluation of Performance (LEP)

CONTEXT

The evaluation of competence in the health professions is of great importance to the public and professionals alike. Recent efforts to design dependable and accurate systems of assessment for demanding clinical roles are increasingly attempting to focus on all-round competence of practitioners. Many challenges are faced in this field as a balance between robust assessment methodology and feasibility in practice is crucial to implementation and adoption.

OBJECTIVES

The authors discuss some of the challenges faced by educators and clinicians involved in the development of systems of assessment for the health professions, and describe a method which aims to address these issues in the assessment of postgraduate dental training in Scotland.

DISCUSSION

Three of the major challenges facing educators and clinicians involved in the design of competency-based systems of assessment are considered: the requirement for evaluation in different areas of competence; the importance of association of assessment with the training objectives, and the types and focus of the assessment introduced. Issues around the use of formative and summative assessment, and the perception that these must always remain completely separate, are discussed in detail.

SOLUTIONS

A proposal is made for the introduction of a method of assessment which has been designed keeping these challenges in mind. The rationale behind this assessment method is described.