Assessing the competencies in general surgery residency training

Postgraduate Assessment Approaches Across Surgical Specialties: A Systematic Review of the Published Evidence

PURPOSE

Ensuring competence for surgical trainees requires holistic assessment of the qualities and competencies necessary to practice safely and effectively. To determine the next steps toward achieving this aim, the authors conducted a systematic review to summarize and appraise the available evidence related to any assessment approach to postgraduate surgical training and to identify the dominant themes for assessment approaches across different specialties or countries.

METHOD

Medline and Embase were searched on January 10, 2019, without language or time restrictions. Any peer-reviewed study that described an assessment framework (in practice or novel) throughout postgraduate surgical training globally was included. An iterative review and thematic analysis were performed on full-text articles to determine assessment themes. Studies were then grouped by assessment themes. A tailored quality assessment of the studies included in the final analysis was conducted. Assessment themes and validity were compared across surgical specialties and countries.

RESULTS

From an initial 7,059 articles, 91 studies (evaluating 6,563 surgical trainees) were included in the final analysis. Ten defined assessment themes were extracted. Ten studies (11.0%) were deemed low risk of bias based on the quality assessment tool used and thus were determined to be high quality. Minor differences in assessment themes were observed between specialties and countries. Assessment themes neglected by individual surgical specialties and assessment themes that need validated assessment tools were identified.

CONCLUSIONS

This review highlights the low quality of evidence and fragmented efforts to develop and optimize surgical assessments. The minor differences observed demonstrate a common approach, globally and across specialties, related to surgical assessments. A paradigm shift in assessment approaches, which will require national and international collaboration, is required to optimize design and validation so that a comprehensive assessment of surgical competence can be implemented.

What is an hour-lecture worth?

BACKGROUND

Although there are many ways to convey knowledge, attitudes, and techniques when teaching residents and students, the most optimal method (lecture, online lecture, online tutorial, simulator practice, and so on) is yet to be determined.

METHODS

This study was designed to be a prospective analysis of change in resident behavior, and the model chosen was resident compliance with alcohol screening during admissions to the trauma service. Baseline values were determined the month before the educational "intervention," which was planned to be a 1-hour lecture during Grand Rounds on the importance of screening for alcohol disuse syndromes. After the "intervention," results were analyzed at 3 points in time: during the first month after the lecture and then at 3 and 12 months.

RESULTS

Resident compliance with alcohol usage screening rose from 53% at baseline to 80% at 1 year.

CONCLUSIONS

This straightforward model of utility of a lecture showed a significant change in resident behavior.

Assessing the competencies in general surgery residency training

BACKGROUND

Accreditation Council for Graduate Medical Education (ACGME) competencies have increased the focus on education. A 1-year study of observational assessments was conducted.

METHODS

"Point of Observation" evaluations were completed by faculty for postgraduate year (PGY) I and II surgery residents. Resident procedures and patient office visits were rated in 9 categories, using a Likert scale. Interns were expected to perform at a novice level (0-30%) and PGY II residents at an advanced level, with improvements expected.

RESULTS

PGY I and II residents showed overall improved scores (12% and 6%, respectively) over 1 year. Intern improvements in 9 categories were linear, ranging from 0% to 48%. PGY II resident scores were more variable, with improvements noted in 6 categories ranging from 10% to 30%. Three categories declined in scores, ranging from 2% to 18%.

CONCLUSIONS

Competencies bring emphasis to education and measurable outcomes. Early efforts have proven valuable in identifying curriculum and learning needs.