Strategies for increasing the feasibility of performance assessments during competency-based education: Subjective and objective evaluations correlate in the operating room

A scoping review of assessment methods of competence of general surgical trainees

BACKGROUND

Only rigorous evaluation of competence will result in the production of safe surgeons that are able to provide the best care for patients. The development of competency-based assessment should ultimately be evidence driven.

OBJECTIVES

Explore the volume of existing evidence pertaining to the different objective assessment methods reported in the literature.

ELIGIBILITY CRITERIA

Studies describing objective assessment of postgraduate general surgical trainees within the last 20 years.

SOURCES OF EVIDENCE

PubMed, Ovid Medline and Web of Sciences.

CHARTING METHODS

A data chart proforma was designed and data were extracted into tables. Basic numerical analysis of extracted data and narrative synthesis of charted data.

RESULTS

A total of 343 papers were reviewed. 26 were eligible for inclusion. 92% of articles were published from 2008 onwards. 50% have been published in the last five years. The articles originated from 6 different countries, predominantly the United Kingdom (42%), followed by the United States of America (38%). In addition, a small number were published from Canada (8%), Japan (4%), Germany (4%) and Australia (4%). UK publications were predominantly between 2008 and 2014 while the USA had a later predominance between 2015 and 2018. 42% were based on quantitative methodology, 27% had a qualitative approach while 31% had mixed analysis. There were sixteen assessment methods presented. The most common type of assessment was Objective Structured Assessments (27%), which included Objective Structured Assessment of Technical Skill (OSATS) (23%) and Objective Structured Assessment of Non-Technical Skill (4%). Procedure Based Assessment (PBA) (23%) and Entrustability Scales (23%) were also prevalent.

CONCLUSIONS

This scoping review has identified a range of different assessment methods. The assessment methods with a higher volume and level of supporting evidence were OSATS, PBAs and Entrustability Scales. There was a lower volume and level of supporting evidence found within this review for the remaining assessment methods.

Sales training and ICT: a literature review

Purpose

The purpose of this paper is to identify in the specialized literature published in the past 20 years about sales training, some elements or key factors that could be applied to the use of information and communication technologies (ICT) as a support for the learning of the detection of business opportunities.

Design/methodology/approach

This is a study of a documentary nature in which a method of systematic literature review was applied to 132 studies published in journals indexed in Scopus and Scielo on sales training processes. The analysis of the data combined a process of categorization and frequencies statistical analysis.

Findings

The results highlight the role of ICT and specifically the use of mobile devices to support the strengthening of training in sales in terms of relationship, opportunity and flexibility in communication with the client, the construction of positive emotional environments and experiential/situated training.

Originality/value

The originality of this study lies in the extraction and interpretation of key factors focused on addressing a guiding question about the relationships between the use of ICT and sales training.

How Do Thresholds of Principle and Preference Influence Surgeon Assessments of Learner Performance?

OBJECTIVE

The present study asks whether intraoperative principles are shared among faculty in a single residency program and explores how surgeons' individual thresholds between principles and preferences might influence assessment.

BACKGROUND

Surgical education continues to face significant challenges in the implementation of intraoperative assessment. Competency-based medical education assumes the possibility of a shared standard of competence, but intersurgeon variation is prevalent and, at times, valued in surgical education. Such procedural variation may pose problems for assessment.

METHODS

An entire surgical division (n = 11) was recruited to participate in video-guided interviews. Each surgeon assessed intraoperative performance in 8 video clips from a single laparoscopic radical left nephrectomy performed by a senior learner (>PGY5). Interviews were audio recorded, transcribed, and analyzed using the constant comparative method of grounded theory.

RESULTS

Surgeons' responses revealed 5 shared generic principles: choosing the right plane, knowing what comes next, recognizing normal and abnormal, making safe progress, and handling tools and tissues appropriately. The surgeons, however, disagreed both on whether a particular performance upheld a principle and on how the performance could improve. This variation subsequently shaped their reported assessment of the learner's performance.

CONCLUSIONS

The findings of the present study provide the first empirical evidence to suggest that surgeons' attitudes toward their own procedural variations may be an important influence on the subjectivity of intraoperative assessment in surgical education. Assessment based on intraoperative entrustment may harness such subjectivity for the purpose of implementing competency-based surgical education.