Validation of a novel intraoperative assessment tool: The Surgical Procedure Feedback Rubric

Trained to cut? A literature review of veterinary surgical resident training

A surgical residency trains veterinary graduates to a higher level of expertise in surgical procedures than is possible during veterinary school and prepares a resident to pursue board certification in surgery. The education of veterinary surgical residents has changed minimally since its inception in the twentieth century, and there are insufficient studies to determine if residency programs are producing surgeons with competence in each of the necessary procedural categories. The aims of this review were to report the current theory and methods used to provide surgical education to residents, to discuss the training most likely to create a competent, board-certified surgeon and to review assessment methods used during training. Several literature searches using broad terms such as "veterinary surgery residency," "veterinary surgery resident," and "veterinary surgical training" were performed using PubMed, CAB abstracts, and Google Scholar. Literature pertinent to theory, methods, training, and assessment of veterinary surgical residents was included. The reviewed literature demonstrated the need for research-based learning curves for specific procedures. Simulation training is known to facilitate deliberate practice and should be leveraged where possible to reach competency. The creation of validated assessment methods should be pursued as it enables assessment of competency instead of inferring its development from case logs. Understanding and supporting learner cognition and providing sufficient feedback remain important issues in the field. Surgical educators are urged to continue to search for innovative and evidence-based ways to train competent surgical residents.

Empowering the Resident Learner

Surgical resident autonomy is an essential element of the transition from supervised training to independent practice. However, in an age of duty hour restrictions, legislative constraints, increased litigation, and heightened societal expectations, training an autonomous resident proves increasingly difficult. To tackle these barriers to training successful surgeons, the American College of Surgeons Committee on Resident Education initiated the Resident Mentored Autonomy Project. As a subdivision of this project, the Empowered Learner research team here presents a framework for training the self-directed and empowered surgical resident learner. There are many strategies by which surgical faculty, program directors and chairs, and residents themselves may engage to improve resident operative autonomy.

Minimally Trained Analysts Can Perform Fast, Objective Assessment of Orthopedic Technical Skill from Fluoroscopic Images

Skill assessment in orthopedics has traditionally relied on subjective impressions from a supervising surgeon. The feedback derived from these tools may be limited by bias and other practical issues. Objective analysis of intraoperative fluoroscopic images offers an inexpensive, repeatable, and precise assessment strategy without bias. Assessors generally refrain from using the scores of images obtained throughout the operation to evaluate skill for practical reasons. A new system was designed to facilitate rapid analysis of this fluoroscopy via minimally trained analysts. Four expert and four novice analysts independently measured one objective metric for skill using both a custom analysis software and a commercial alternative. Analysts were able to measure the objective metric three times faster when using the custom software, and without a practical difference in accuracy in comparison to the expert analysts using the commercial software. These results suggest that a well-designed fluoroscopy analysis system can facilitate inexpensive, reliable, and objective assessment of surgical skills.

Evaluating chief resident readiness for the teaching assistant role: The Teaching Evaluation assessment of the chief resident (TEACh-R) instrument

BACKGROUND

The American Board of Surgery has mandated chief residents complete 25 cases in the teaching assistant (TA) role. We developed a structured instrument, the Teaching Evaluation and Assessment of the Chief Resident (TEACh-R), to determine readiness and provide feedback for residents in this role.

METHODS

Senior (PGY3-5) residents were scored on technical and teaching performance by faculty observers using the TEACh-R instrument in the simulation lab. Residents were provided with their TEACh-R scores and surveyed on their experience.

RESULTS

Scores in technical (p < 0.01) and teaching (p < 0.01) domains increased with PGY. Higher technical, but not teaching, scores correlated with attending-rated readiness for operative independence (p 0.02). Autonomy mismatch was inversely correlated with teaching competence (p < 0.01). Residents reported satisfaction with TEACh-R feedback and desire for use of this instrument in operating room settings.

CONCLUSION

Our TEACh-R instrument is an effective way to assess technical and teaching performance in the TA role.

Key dimensions of innovations in workplace-based assessment for postgraduate medical education: a scoping review

BACKGROUND

Specialist training bodies continue to devise innovative methods of gathering information on trainee workplace performance to meet the requirements of competency-based medical education. We reviewed recent innovations in workplace-based assessment (WBA) tools to identify strengths, weaknesses, and trade-offs inherent in their design and use.

METHODS

In this scoping review, using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, we systematically searched databases between 2009 and 2019 for WBA tools with novel characteristics not typically seen in traditional WBAs. These included innovations in rating scales, ways of collecting information, technological innovations, ways of triggering WBAs, and approaches to compiling and using information.

RESULTS

We identified 30 innovative WBA tools whose characteristics could be categorised into seven dimensions: frequency of assessment, granularity (unit of performance assessed), coverage of the curriculum, rating method, initiation of the WBA, information use, and incentives. These dimensions had multiple interdependencies and trade-offs, often balancing generating assessment data with available resources. Philosophical stance on assessment also influenced WBA choice, for example prioritising trainee-centred learning (i.e. initiation of WBA and transparency of assessment data), perceptions of assessment and feedback as burdensome or beneficial, and holistic vs reductionist views on assessment of performance.

CONCLUSIONS

Our synthesis of the literature on innovative WBAs provides a framework for categorising tool characteristics across seven dimensions, systematically teasing apart the considerations in design and use of workplace assessments. It also draws attention to the trade-offs inherent in tool design and selection, and enables a more deliberate consideration of the tool characteristics most appropriate to the local context.

Postgraduate Assessment Approaches Across Surgical Specialties: A Systematic Review of the Published Evidence

PURPOSE

Ensuring competence for surgical trainees requires holistic assessment of the qualities and competencies necessary to practice safely and effectively. To determine the next steps toward achieving this aim, the authors conducted a systematic review to summarize and appraise the available evidence related to any assessment approach to postgraduate surgical training and to identify the dominant themes for assessment approaches across different specialties or countries.

METHOD

Medline and Embase were searched on January 10, 2019, without language or time restrictions. Any peer-reviewed study that described an assessment framework (in practice or novel) throughout postgraduate surgical training globally was included. An iterative review and thematic analysis were performed on full-text articles to determine assessment themes. Studies were then grouped by assessment themes. A tailored quality assessment of the studies included in the final analysis was conducted. Assessment themes and validity were compared across surgical specialties and countries.

RESULTS

From an initial 7,059 articles, 91 studies (evaluating 6,563 surgical trainees) were included in the final analysis. Ten defined assessment themes were extracted. Ten studies (11.0%) were deemed low risk of bias based on the quality assessment tool used and thus were determined to be high quality. Minor differences in assessment themes were observed between specialties and countries. Assessment themes neglected by individual surgical specialties and assessment themes that need validated assessment tools were identified.

CONCLUSIONS

This review highlights the low quality of evidence and fragmented efforts to develop and optimize surgical assessments. The minor differences observed demonstrate a common approach, globally and across specialties, related to surgical assessments. A paradigm shift in assessment approaches, which will require national and international collaboration, is required to optimize design and validation so that a comprehensive assessment of surgical competence can be implemented.

Development of an Assessment Tool for Surgeons in Their First Year of Independent Practice: The Junior Surgeon Performance Assessment Tool

OBJECTIVE

The purpose of this study was to create an assessment tool to evaluate newly practicing surgeons.

DESIGN

In this prospective mixed methods study, a needs assessment was performed by conducting focus groups with practicing general surgeons, asking questions regarding essential surgeon qualities, behaviors observed in inexperienced surgeons, current assessment methods, and desired assessment tool elements and attributes. A qualitative analysis was performed using a grounded theory methodology. The Junior Surgeon Performance Assessment Tool (JSPAT) was created using a 4-point scale for each category developed, with themes identified in the qualitative analysis used to create behavioral anchors. The JSPAT was evaluated by focus group participants and by members of the American College of Surgeons Advisory Council for Rural Surgery using an online survey.

SETTING

Rural and nonuniversity-based hospitals throughout the state of Oregon.

PARTICIPANTS

Practicing general surgeons.

RESULTS

Focus groups consisted of 31 surgeons (mean age 49, mean experience 17 years) from 11 different hospitals. Qualitative analysis revealed 91 different themes, which were grouped into 5 domains (technical skills, interaction with patients, interaction with surgeon colleagues, interactions with the greater medical community, and self-care) to create the assessment tool. Twenty online survey responses providing feedback on the assessment tool were obtained, with 75% rating the JSPAT useful or very useful and 69% satisfied or very satisfied with the time to complete the tool.

CONCLUSIONS

A mixed-methods model was used to create an assessment tool for surgeons in their first year of independent practice. Survey data demonstrated that practicing surgeons find value in the JSPAT.

Surgical education: Lessons from parenthood

Although one might think surgery and parenting have little in common, there are clear parallels. Historically there has been little formal education for either role. Educators and parents relied on modelling the behavior of others, or trial and error techniques. Mentorship and role models have played a critical role in professional development and continue to have a profound impact. Over the past two decades there has been a marked increase in the resources that are available. Coaching, debriefing, deliberate practice, and formal training are now incorporated in residency programs. Specific lessons from parenthood that can be applied to surgical education include: providing a framework, learning through graduated responsibility, communicating expectations, creating a culture, setting the example, encouraging resilience, promoting autonomy, providing feedback, and navigating failure. The final lesson from parenthood: trust that you have taught them well. And you have to let them go.