The Objective Structured Clinical Examination: three decades of development

Comparing an In-Person and Online Continuing Education Intervention to Improve Professional Decision-Making: A Mixed Methods Study

Purpose: This paper compares two iterations (in-person and online) of a multi-stage continuing education program for improving high-risk decision-making among mental health workers. Methods: The mixed-methods study analyzed the following: (1) physiological and psychological arousal during simulated patient interviews; (2) physiological and psychological arousal recorded during real-time decision-making over four months; and (3) thoughts on the process and outcomes of the intervention raised in reflective interviews. Findings: Quantitatively, there were no statistical differences in stress measures between in-person and online simulated interviews or decision-making logs, suggesting they were effective in eliciting reactions commonly found in challenging clinical situations. Qualitatively, participants in both iterations indicated that the intervention caused them to reflect on practice, consider a wider range of factors related to the decisions, and enact approaches to improve decision-making. Conclusions: A carefully constructed online continuing education experience can result in outcomes for experienced social workers that are equivalent to an in-person iteration.

Assessment and Implementation of WOAH Day 1 Competencies (AID-1C): a cyclical methodology for curriculum harmonization with international standards

Introduction

The World Organisation for Animal Health (WOAH) Day 1 Competencies for Graduating Veterinarians provide a standard framework to guide Veterinary Educational Establishments (VEEs) in improving their veterinary public health and population medicine curricula. However, pursuing a curriculum revision to incorporate these standards may be daunting, especially for institutions with limited resources or experience. This manuscript describes a methodology for targeted curriculum revision specifically focused on the WOAH Day 1 Competencies.

Phases of the AID-1 process

The Assessment and Implementation of WOAH Day 1 Competencies (AID-1C) is a six-step, cyclical, collaborative methodology that encompasses a series of tools and processes that help a VEE to evaluate their curriculum, identify and prioritize gaps, and develop and implement an action plan based on the results. The six phases of the AID-1C process include: (1) Assessment of the proficiency of the VEE's graduates in Day 1 Competencies using a structured Evaluation Tool; (2) A systematic curricular review and evaluation; (3) Identification and prioritization of interventions through a group problem-solving and prioritization exercise called Focus Forward; (4) Design and development of interventions to address identified gaps; (5) Curricular implementation; and (6) Monitoring and evaluation. The AID-1C methodology relies upon active involvement of senior students, recent graduates, faculty, instructional staff, and employers throughout the process.

Conclusion

The AID-1C methodology provides a systematic, participatory, collaborative approach that simplifies the planning and execution of the curricular revision, making a complex process more manageable. This enables VEEs to improve their curricula, while moving toward harmonization with WOAH standards. The result is a curriculum that allows a VEE to train well-rounded and competent veterinarians, with the requisite skills to support the veterinary services in their country.

Remote Assessment of Veterinary Clinical Skills Courses During the COVID-19 Pandemic

In spring 2020, the COVID-19 pandemic forced educators to adjust the delivery and assessment of curriculum. While didactic courses moved online, laboratory courses were not amenable to this shift. In particular, assessment of clinical skills courses through common methods including objective structured clinical examinations (OSCEs) became inadvisable. This article describes decisions made for first-, second-, and third-year veterinary students (n = 368) with respect to clinical skills at one US college. This includes the remote completion of a surgical skills curriculum using instructional videos and models and the delaying of laboratory sessions deemed impossible to deliver remotely. First- and third-year students were subsequently assessed using modified remote OSCEs. Second-year students were assessed using the standard surgical skills examination, video-recorded. All first- and third-year students successfully passed their OSCE upon either first attempt or remediation. Two second-year students failed their remediation examination and were offered additional faculty tutoring and another remediation attempt at the start of the fall semester. The remediation rate on the surgical skills examination was not different from that of previous years. One incident of suspected academic dishonesty occurred in the first-year OSCE. Students learned surgical skills successfully at home by practicing on models and receiving feedback of their skills on video recordings. While disappointing, one case of academic dishonesty among the 368 total students tested was not surprising. Remote assessment using modified OSCEs and surgical skills exams appears feasible and fair when in-person testing is not possible.

Communication skills training and assessment of food animal production medicine veterinarians: A component of a voluntary Johne's disease control program

In food animal production medicine (FAPM), the success of control programs for infectious diseases that have serious animal health and economic consequences frequently rely on the veterinarian's effective communication and producer adherence to veterinary recommendations. However, little research has been conducted on communication skills of practicing FAPM veterinarians. During this study, we developed a communication training workshop intervention to support the Atlantic Johne's Disease Initiative. Seventeen FAPM veterinarians across 10 clinics practicing within Maritime Canada participated in a pre-post intervention study design. Communication skills were evaluated utilizing 3 assessment tools; an objective structured clinical exam (OSCE), standardized client feedback, and an instrument designed for veterinary participants to assess their self-efficacy. Study results showed that before training, communication skills of participating veterinarians had limitations, including skill deficits in communication tasks strongly associated with increased adherence to veterinary recommendations. Based on the 3 assessment tools, communication skills of participating veterinarians improved with the training provided. Significant increases were detected in pre- to postintervention self-efficacy percentage scores, OSCE percentage and global scores from expert raters, and OSCE percentage and global scores from standardized client feedback. These improvements emphasize the importance of communication skills training specific to FAPM.

The objective structured clinical examination as an assessment strategy for clinical competence in novice nursing practitioners in Taiwan

Background

The conventional written tests and professional assessment have limitation in fair judgement of clinical competence. Because the examiners may not have total objectivity and may lack standardization throughout the assessment process. We sought to design a valid method of competence assessment in medical and nursing specialties. This work was aimed to develop an Objective Structured Clinical Exam (OSCE) to evaluate novice nursing practitioners’ clinical competency, work stress, professional confidence, and career satisfaction.

Methods

A Quasi-experimental study (pre-post). Fifty-five novice nursing practitioners received the OSCE three-months following their graduation, which consisted of four stations: history taking, physical examination, problem-directed management, interpersonal communication, and the required techniques of related procedures. The examiners had to complete an assessment checklist, and the participants had to complete a pre-post questionnaire (modified from a Nursing Competency Questionnaire, a Stress scale, and Satisfaction with Learning scale).

Results

Among the novice nursing practitioners, 41 of them (74.5 %) passed the exam with a mean score of 61.38 ± 8.34. There was a significantly higher passing rate among nurses who were working in medical-surgical wards (85.7 %) and the intensive care unit-emergency department (77.8 %) compared to novice nursing practitioners working in other units. All the novice nursing practitioners at Station A had poor performance in assessing patients with a fever. OSCE performance was more associated with educational attainment and work unit, rather than the gender. Finally, the participants showed statistically significant increases in their clinical competency, confidence in their professional competence, satisfaction with the clinical practice, and decreased work stress after the OSCE.

Conclusions

We found that the OSCE process had a positive educational effect, in providing a meaningful and accurate assessment of the competence of novice nursing practitioners. An appropriate OSCE program is vital for novice nursing practitioners, educators, and administrators. The effective application of OSCEs can help novice nursing practitioners gain confidence in their clinical skills.

Comparison of Self-Directed and Instructor-Led Practice Sessions for Teaching Clinical Skills in Food Animal Reproductive Medicine

While the use of simulator-based clinical skill training has become increasingly popular in veterinary education in recent years, little research has been done regarding optimal implementation of such tools to maximize student learning in veterinary curricula. The objective of this study was to compare the effects of supervised and unsupervised deliberate practice on clinical skills development in veterinary medicine students. A total of 150 veterinary students took part in instructor-led practice (supervised) or self-directed practice (unsupervised) at a selection of four learning stations in a veterinary skills laboratory. Each learning station consisted of a teaching simulator, materials required to complete the task, and a standard operating procedure detailing how to execute the task. Students used Likert scales to self-evaluate their clinical skills before and after practice sessions, in addition to evaluating their motivation to practice a given task. An objective structured clinical examination (OSCE) was used to compare participants' clinical skills performance between learning stations. We were able to show that practice had a significant positive effect on OSCE scores at three out of six available learning stations. Motivation ratings varied between learning stations and were positively correlated with an increase in self-perceived clinical skills. At an instructor-to-student ratio of approximately 1:8, supervision had no effect on OSCE scores at four out of six learning stations. At the remaining two learning stations, self-directed practice resulted in better learning outcomes than instructor-led practice.

Using Creativity as an Educational Tool in Veterinary Surgery: Students' Perceptions and Surgical Performance

With the aim of improving students' ability to handle the complexity of surgery, we introduced a creative assignment in a veterinary surgical course. We hypothesized that by using this active, inductive educational method, reflection, creativity and self-efficacy in student novice surgeons could be improved. During a companion animal surgical course an intervention group was investigated against a control group. Twenty-nine fourth-year students were instructed in ovariohysterectomy by classical lectures, while 23 fourth-year students were provided with creative materials and assigned to consider and illustrate how to perform the procedure themselves. Surgical performance was assessed for both groups using a modified Objective Structured Assessment of Technical Skills (OSATS) while performing a simulated ovariohysterectomy. Furthermore, both groups were investigated with respect to how they would handle a specific hypothetical surgical complication. Semi-structured interviews were conducted with 17 intervention-group students and were analyzed using thematic analysis. The intervention group showed a significantly better performance and needed significantly less help with the surgical complication than the control group students. Data from interviews furthermore demonstrated that students believed the creative intervention produced increased reflection, more creative initiatives, and a feeling of security before surgery. Our study results thus indicate that an educational tool which stimulates creative thinking can promote reflection, creativity, and self-efficacy in novice surgeons without compromising surgical performance.

Ability of a bovine transrectal palpation objective structured clinical examination to predict veterinary students' pregnancy diagnosis accuracy

Bovine pregnancy diagnosis (PD) by transrectal palpation (TRP) is one of the most frequently performed procedures in bovine practice, and an important competency for veterinary graduates. It is currently not known if pre-existing TRP skills on non-pregnant cows can be used to predict students' future PD accuracy. The study objective was to evaluate if TRP objective structured clinical examination (OSCE) scores can predict students' future PD accuracy.Fourth year (of a six-year programme) veterinary students (n=128) received TRP and PD training on Breed'n Betsy (BB) simulators and live cows. Students' TRP skills were assessed using a live cow TRP OSCE after completion of the fourth year training. The same students received additional TRP (BB and live cows) and PD (BB) training sessions in the first semester of their fifth year. PD accuracy was assessed after the additional TRP and PD training, five months after the TRP OSCE assessment and measured as sensitivity and specificity (the ability to correctly identify the presence and absence of pregnancy, respectively). Each student palpated six cows transrectally to diagnose pregnancy status and stage for the PD assessment. The TRP OSCE results were analysed as predictors for students' PD accuracy.Students with 'competent palpation skills' on the TRP OSCE had higher PD specificity. The individual OSCE components that were predictive of higher PD accuracy were students' ability to estimate ovarian size, identify uterine position and exclude intrauterine fluid. It was concluded that a TRP OSCE has the ability to predict students' future PD accuracy.

Why So Stressed? A Descriptive Thematic Analysis of Physical Therapy Students' Descriptions of Causes of Anxiety during Objective Structured Clinical Exams

Purpose: The purpose of this study was to collect and conduct a descriptive content analysis of the primary triggers of practical exam anxiety in Master of Physical Therapy (MPT) students in a Canadian university programme. Method: First and second-year MPT students were invited to reflect upon their top 5 sources or triggers of OSCE exam anxiety, collected in written format during a low-stress, low-examination period of their programme. All participants had participated in at least 3 OSCEs before providing data. The emergent themes were member-checked with 10 of the original participants to improve trustworthiness of the results. Results: 56 of a possible 105 students provided 224 triggers of OSCE anxiety. Thematic content analysis revealed 6 emergent meta-themes that adequately captured all triggers. They were: social performance anxiety, fear of lacking competence, overvaluing the outcome, fear of the unknown, impaired personal health/coping resources, and operational/procedural influences. These meta-themes were endorsed by the participant sub-group. Conclusions: OSCEs are common forms of evaluation in MPT training programmes, but are also highly anxiogenic. The first step toward mitigating exam anxiety, thereby ensuring exam performance is less confounded by anxiety, is to identify the common triggers. Confidence in results will be strengthened by replication in other programmes.

Evaluation of a coaching workshop for the management of veterinary nursing students' OSCE-associated test anxiety

Background

High stress levels amongst undergraduates (particularly in relation to assessment) and efforts to improve mental wellbeing have been increasingly reported in the veterinary educational literature. However reports to date have primarily focused on the experiences of students of veterinary medicine, rather than veterinary nursing students.

Methods

The purpose of this mixed method sequential explanatory study was to establish the "Big-five" personality traits and quantify the level of test anxiety associated with objective structured clinical examinations (OSCEs) amongst a cohort of 23 final year veterinary nursing students at an Irish third level college. The 12 item Brief FRIEDBEN Test Anxiety Scale (B-FTAS) and the 20 item mini International Personality Item Pool (mini-IPIP) were used to identify test anxiety levels and personality traits in this cohort. Focus groups were then employed to examine the effectiveness of a coaching intervention in ameliorating this test anxiety.

Results

The initial, quantitative, phase found these students to have higher levels of test anxiety than previously reported for undergraduates sitting written examinations. No association was found between test anxiety and neurotic personality traits in this student cohort. In the qualitative follow up phase the coaching intervention was reported to have been helpful in equipping the students to better manage test anxiety. The OSCE stressors identified in this study closely resembled those previously reported by nursing and midwifery students.

Conclusions

The shared experience of the coaching intervention and formative OSCE was reported to have been helpful in empowering the students to manage assessment-associated anxiety. Implications and recommendations for educators were identified.

Appraising the role of the virtual patient for therapeutics health education

BACKGROUND

Face-to-face instruction, paper-based case-studies and clinical placements remain the most commonly used teaching methods for therapeutics curricula. Presenting clinical content in a didactic manner presents challenges in engaging learners and developing their clinical reasoning skills which may be overcome by inclusion of the virtual patient (VP). Currently there is limited literature examining the use of the VP in therapeutics teaching and learning. This review aimed to determine the role of VPs in therapeutics education, specifically the impact on student experiences, performance, and clinical skills.

METHODS

A search of primary literature was conducted with search terms including virtual patient, education, health, AND learning. Boolean operators were applied to include studies from health relevant fields with article titles and abstracts vetted.

RESULTS

Nine of the 21 included studies were control-matched, and all but one compared VPs to traditional teaching. VPs enhanced the learning experience in all 17 studies that measured this outcome. Fourteen studies measured performance and clinical skills and 12 found VPs were beneficial, while two did not. The VP was not superior to traditional teaching in all studies, but the VP appeared beneficial to the student learning experience. Discrepancy was found between the impact of VPs on short- and long-term knowledge.

IMPLICATIONS

The VP appears to enhance the student learning experience and has a role in therapeutics education, however a blended-learning (BL) approach may be required to account for individual learning styles. Additional investigation is required to clarify the efficacy of the VP, particularly as a component of BL, on longer-term knowledge retention.

On the Assessment of Paramedic Competence: A Narrative Review with Practice Implications

Introduction

Paramedicine is experiencing significant growth in scope of practice, autonomy, and role in the health care system. Despite clinical governance models, the degree to which paramedicine ultimately can be safe and effective will be dependent on the individuals the profession deems suited to practice. This creates an imperative for those responsible for these decisions to ensure that assessments of paramedic competence are indeed accurate, trustworthy, and defensible.

Purpose

The purpose of this study was to explore and synthesize relevant theoretical foundations and literature informing best practices in performance-based assessment (PBA) of competence, as it might be applied to paramedicine, for design or evaluation of assessment programs.

Methods

A narrative review methodology was applied to focus intentionally, but broadly, on purpose relevant, theoretically derived research that could inform assessment protocols in paramedicine. Primary and secondary studies from a number of health professions that contributed to and informed best practices related to the assessment of paramedic clinical competence were included and synthesized.

Results

Multiple conceptual frameworks, psychometric requirements, and emerging lines of research are forwarded. Seventeen practice implications are derived to promote understanding as well as best practices and evaluation criteria for educators, employers, and/or licensing/certifying bodies when considering the assessment of paramedic competence.

Conclusions

The assessment of paramedic competence is a complex process requiring an understanding, appreciation for, and integration of conceptual and psychometric principles. The field of PBA is advancing rapidly with numerous opportunities for research.

TavaresW,BoetS.On the assessment of paramedic competence: a narrative review with practice implications.Prehosp Disaster Med.2016;31(1):64–73.

Does a Rater's Professional Background Influence Communication Skills Assessment?

There is increasing pressure in veterinary education to teach and assess communication skills, with the Objective Structured Clinical Examination (OSCE) being the most common assessment method. Previous research reveals that raters are a large source of variance in OSCEs. This study focused on examining the effect of raters' professional background as a source of variance when assessing students' communication skills. Twenty-three raters were categorized according to their professional background: clinical sciences (n=11), basic sciences (n=4), clinical communication (n=5), or hospital administrator/clinical skills technicians (n=3). Raters from each professional background were assigned to the same station and assessed the same students during two four-station OSCEs. Students were in year 2 of their pre-clinical program. Repeated-measures ANOVA results showed that OSCE scores awarded by the rater groups differed significantly: (F(matched_station_1) [2,91]=6.97, p=.002), (F(matched_station_2) [3,90]=13.95, p=.001), (F(matched_station_3) [3,90]=8.76, p=.001), and ((Fmatched_station_4) [2,91]=30.60, p=.001). A significant time effect between the two OSCEs was calculated for matched stations 1, 2, and 4, indicating improved student performances. Raters with a clinical communication skills background assigned scores that were significantly lower compared to the other rater groups. Analysis of written feedback provided by the clinical sciences raters showed that they were influenced by the students' clinical knowledge of the case and that they did not rely solely on the communication checklist items. This study shows that it is important to consider rater background both in recruitment and training programs for communication skills' assessment.

The use of global rating scales for OSCEs in veterinary medicine

OSCEs (Objective Structured Clinical Examinations) are widely used in health professions to assess clinical skills competence. Raters use standardized binary checklists (CL) or multi-dimensional global rating scales (GRS) to score candidates performing specific tasks. This study assessed the reliability of CL and GRS scores in the assessment of veterinary students, and is the first study to demonstrate the reliability of GRS within veterinary medical education. Twelve raters from two different schools (6 from University of Calgary [UCVM] and 6 from Royal (Dick) School of Veterinary Studies [R(D)SVS] were asked to score 12 students (6 from each school). All raters assessed all students (video recordings) during 4 OSCE stations (bovine haltering, gowning and gloving, equine bandaging and skin suturing). Raters scored students using a CL, followed by the GRS. Novice raters (6 R(D)SVS) were assessed independently of expert raters (6 UCVM). Generalizability theory (G theory), analysis of variance (ANOVA) and t-tests were used to determine the reliability of rater scores, assess any between school differences (by student, by rater), and determine if there were differences between CL and GRS scores. There was no significant difference in rater performance with use of the CL or the GRS. Scores from the CL were significantly higher than scores from the GRS. The reliability of checklist scores were .42 and .76 for novice and expert raters respectively. The reliability of the global rating scale scores were .7 and .86 for novice and expert raters respectively. A decision study (D-study) showed that once trained using CL, GRS could be utilized to reliably score clinical skills in veterinary medicine with both novice and experienced raters.

Evaluation of a training model to teach veterinary students a technique for injecting the jugular vein in horses

In this study, a newly-developed model for training veterinary students to inject the jugular vein in horses was evaluated as an additional tool to supplement the current method of teaching. The model was first validated by 19 experienced equine veterinarians, who judged the model to be a realistic and valuable tool for learning the technique. Subsequently, it was assessed using 24 students who were divided randomly into two groups. The injection technique was taught conventionally in a classroom lecture and a live demonstration to both groups, but only group 1 received additional training on the new model. All participants filled out self-assessment questionnaires before and after group 1 received training on the model. Finally, the proficiency of both groups was assessed using an objective structured clinical evaluation (OSCE) on live horses. Students from group 1 showed significantly improved confidence after their additional training on the model and also showed greater confidence when compared to group 2 students. In the OSCE, group 1 had a significantly better score compared to group 2: the median (with inter-quartile range) was 15 (0.7) vs. 11.5 (2.8) points out of 15, respectively. The training model proved to be a useful tool to teach veterinary students how to perform jugular vein injections in horses in a controlled environment, without time limitations or animal welfare concerns. The newly developed training model offers an inexpensive, efficient, animal-sparing way to teach this clinical skill to veterinary students.

Assessment of first-year veterinary students' communication skills using an objective structured clinical examination: the importance of context

Communication skills are considered to be a core clinical skill in veterinary medicine and essential for practice success, including outcomes of care for patients and clients. While veterinary schools include communication skills training in their programs, there is minimal knowledge on how best to assess communication competence throughout the undergraduate program. The purpose of this study was to further our understanding of the reliability, utility, and suitability of a communication skills Objective Structured Clinical Examination (OSCE). Specifically we wanted to (1) identify the greatest source of variability (student, rater, station, and track) within a first-year, four station OSCE using exam scores and scores from videotape review by two trained raters, and (2) determine the effect of different stations on students' communication skills performance. Reliability of the scores from both the exam data and the two expert raters was 0.50 and 0.46 respectively, with the greatest amount of variance attributable to student by station. The percentage of variance due to raters in the exam data was 16.35%, whereas the percentage of variance for the two expert raters was 0%. These results have three important implications. First, the results reinforce the need for communication educators to emphasize that use of communication skills is moderated by the context of the clinical interaction. Second, by increasing rater training the amount of error in the scores due to raters can be reduced and inter-rater reliability increases. Third, the communication assessment method (in this case the OSCE checklist) should be built purposefully, taking into consideration the context of the case.

Workplace-based assessment instruments in the health sciences

A historical overview of the development of assessment instruments in the health sciences is presented here, with specific attention paid to workplace-based assessment instruments. Three instruments are reviewed in detail: the mini clinical evaluation exercise (mCEX), direct observation of procedural skills (DOPS), and multi-source feedback (MSF). Features common to these instruments include their authenticity, their use in assessing professional skills, and the opportunities they afford for the provision of feedback. Although almost exclusively used in graduate medical training, they are likely to play an increasingly important role in the assessment of veterinary undergraduate students in preparation for professional practice. However, the time and cost associated with implementing these instruments raises questions about their feasibility. The continued search for the holy grail of assessment instruments and the challenges relating to the need for trained assessors leads us to conclude that ultimately, the competence of health professionals should continue to be measured using several complementary instruments.

Assessment of technical skills: best practices

Assessment is an important aspect of veterinary education from the point of view of setting standards, driving learning, providing feedback, and reassuring society that veterinarians are competent to assume the responsibilities entrusted to them. However, no single format exists that can, by itself, assess the complex mixture of knowledge and skills essential to the veterinarian's role. The areas that are most challenging to assess are those involving behaviors and attitudes. These include the various technical skills required for diagnosis and treatment. One approach, aimed at retaining validity but improving reliability compared with traditional, more subjective methods, first described in medicine 35 years ago, is the Objective Structured Clinical Examination (OSCE), which has been introduced into veterinary education as the Objective Structured Practical Veterinary Examination (OSPVE) and run at the Royal Veterinary College since 2004. This approach is good for the assessment of competence in relation to isolated techniques and whole procedures but has been criticized for the way in which these are tested out of context. However, further development of structured clinical assessments, such as the mini-Clinical Examination and the Direct Observation of Procedural Skills, may help address some of these limitations, and the use of multi-source feedback, particularly client feedback, may allow the further domains of professional behaviors, attitudes, and communication to be judged and developed.

Assessment of first-year veterinary students' clinical skills using objective structured clinical examinations

The DVM program at the University of Calgary offers a Clinical Skills course each year for the first three years. The course is designed to teach students the procedural skills required for entry-level general veterinary practice. Objective Structured Clinical Examinations (OSCEs) were used to assess students' performance on these procedural skills. A series of three OSCEs were developed for the first year. Content was determined by an exam blueprint, exam scoring sheets were created, rater training was provided, a mock OSCE was performed with faculty and staff, and the criterion-referencing Ebel method was used to set cut scores for each station using two content experts. Each station and the overall exam were graded as pass or fail. Thirty first-year DVM students were assessed. Content validity was ensured by the exam blueprint and expert review. Reliability (coefficient α) of the stations from the three OSCE exams ranged from 0.0 to 0.71. The three exam reliabilities (Generalizability Theory) were, for OSCE 1, G=0.56; OSCE 2, G=0.37; and OSCE 3, G=0.32. Preliminary analysis has suggested that the OSCEs demonstrate face and content validity, and certain stations demonstrated adequate reliability. Overall exam reliability was low, which reflects issues with first-time exam delivery. Because this year was the first that this course was taught and this exam format was used, work continues in the program on the teaching of the procedural skills and the development and revision of OSCE stations and scoring checklists.

Assessing competence in emergency medicine trainees: an overview of effective methodologies

How do we define competence in emergency medicine (EM), and how do we know when a resident has achieved it? In recent years, the idea of physician competence has become widely recognized as being multidimensional. This has resulted in an emphasis on competency-based education and assessment. We describe an up-to-date model to assess competence in EM. An overview of appropriate EM assessment tools is provided, along with their significant strengths and limitations. Sample behaviours representative of core competencies commonly assessed in EM training are matched to appropriate assessment tools. This review may serve as an introductory resource for EM clinicians, teachers and educators involved in EM trainee assessment.

Current methods in use for assessing clinical competencies: what works?

An online survey was used to capture qualitative descriptions of methods used by a veterinary college to assess clinical competencies in its students. Each college was specifically asked about use of the methods detailed in the Toolbox of Assessment Methods developed by the Accreditation Council for Graduate Medical Education and the American Board of Medical Specialties. Additionally, each college was asked to detail the methods used to ensure competency in each of the nine areas specified by the American Veterinary Medical Association Council on Education. Associate deans of academic affairs or their equivalents at veterinary colleges in the United States, the United Kingdom, Canada, and the Caribbean were contacted by e-mail and asked to complete the survey. Responses were obtained from 24 of 32 colleges. The methods most often used were review of students' medical records (16), checklist evaluation of must-learn skills (16), procedural logs (11), multiple-choice skill examinations (11), case simulations using role-playing (7), short-answer skill examinations (7), global rating of live or recorded performance (7), case simulations using computerized case simulations (7), 360-degree evaluation of clinical performance (4), and standardized patient or client examination (3). Additional methods used included medical record portfolio review, paper-and-pencil branching problems, chart-stimulated oral exams, externship mentor evaluation, performance rubrics for clinical rotations, direct observation and query on cases, video evaluation, case correlation tasks, and an employer survey. Non-realistic models were used more often for skill evaluation than realistic models. One college used virtual-reality models for testing.

Objective structured clinical examinations (OSCEs) as a summative evaluation tool in a ruminant health management rotation for final-year DVM students

The objective structured clinical examination (OSCE) has been used for 10 years at the Ontario Veterinary College, University of Guelph, to evaluate the clinical competencies in ruminant health management of final-year DVM students. The performance of these students in the summative assessment, which includes the use of OSCEs, was compared to their formative assessment, given at the end of the rotation. Specifically, classification of students' performance as poor (bottom 10% of the grade range versus "serious deficits") or superior ("A grade" versus "exceeds expectations") was compared. Agreement between the two types of assessment is slight, regardless of whether assessing diagnostic process skills or technical skills--and regardless of whether all students were assessed or only those enrolled in food-animal or mixed streams in their final year--which suggests that the two assess different types of skills. OSCEs are a useful and viable tool for objectively assessing clinical skills in ruminant health management.