Consultant surgeons' opinion of the skills required of basic surgical trainees

Generative artificial intelligence in surgery

Generative artificial intelligence is able to collect, extract, digest, and generate information in an understandable way for humans. As the first surgical applications of generative artificial intelligence are applied, this perspective paper aims to provide a comprehensive overview of current applications and future perspectives for the application of generative artificial intelligence in surgery, from preoperative planning to training. Generative artificial intelligence can be used before surgery for planning and decision support by extracting patient information and providing patients with information and simulation regarding the procedure. Intraoperatively, generative artificial intelligence can document data that is normally not captured as intraoperative adverse events or provide information to help decision-making. Postoperatively, GAIs can help with patient discharge and follow-up. The ability to provide real-time feedback and store it for later review is an important capability of GAIs. GAI applications are emerging as highly specialized, task-specific tools for tasks such as data extraction, synthesis, presentation, and communication within the realm of surgery. GAIs have the potential to play a pivotal role in facilitating interaction between surgeons and artificial intelligence.

The development of a tool to assess medical students' non-technical skills - The Norwegian medical students' non-technical skills (NorMS-NTS)

PURPOSE

New physicians need to master non-technical skills (NTS), as high levels of NTS have been shown to increase patient safety. It has also been shown that NTS can be improved through training. This study aimed to establish the necessary NTS for Norwegian medical students to create a tool for formative and summative assessments.

METHODS

Focus group interviews were conducted with colleagues and patients of newly graduated physicians. Interviews were then analyzed using card sort methods, and the identified NTS were used to establish a framework. Focus groups commented on a prototype of an NTS assessment tool. Finally, we conducted a search of existing tools and literature. The final tool was developed based on the combined inputs.

RESULTS

We created Norwegian medical students' non-technical skills (NorMS-NTS) assessment tool containing four main categories; together comprising 13 elements and a rating scale for the NTS of the person observed.

CONCLUSIONS

The NorMS-NTS represents a purpose-made tool for assessing newly graduated physicians' NTS. It is similar to existing assessment tools but based on domain-specific user perspectives obtained through focus group interviews and feedback, integrated with results from a literature search, and with consideration of existing NTS tools.

Using Job Analysis for Identifying the Desired Competencies of 21st-Century Surgeons for Improving Trainees Selection

OBJECTIVE

The current selection for surgical training is based on ineffective methods. In order to identify or to develop more valid selection tools to improve the selection, it is necessary to first define what are the competencies that are most important for success in contemporary surgery. Therefore, the current study aims to identify what competencies are required for success as a surgeon in the 21st-century and to evaluate their relative importance for selection for surgical training.

METHODS

Job analysis was conducted using a mixed-methods design. First, 104 senior surgeons from all surgical fields from various hospitals in Israel were interviewed in order to query their perceptions of competencies associated with success as a surgeon. Their answers were coded and analyzed to create a list of important competencies. Next, a larger sample of 1,102 surgeons and residents from all surgical fields completed a questionnaire in which they rated the importance of each competency in the list for success as a surgeon and for selection for surgical training in the 21st-century.

RESULTS

Twenty-four competencies (five technical skills, six cognitive abilities, 13 personality characteristics) were identified in the interview analysis. Analysis of the questionnaire's data revealed that all 24 competencies were perceived as important for success as a surgeon in the 21st-century as well as for selection for surgical training. The perceived importance of personality characteristics was higher than both cognitive abilities (p < 0.001) and technical skills (p < 0.001). The results did not differ between different surgical fields.

CONCLUSIONS

Twenty-four competencies were identified as important for 21st-century surgeons and for selection for surgical training. Although all competencies were perceived as important, personality characteristics were perceived as more important than technical skills and cognitive abilities. This updated definition of required competencies may aid in developing more valid selection methods of candidates for surgical training.

Validation of a Newly Developed Competency Assessment Tool for the Posterior Sagittal Anorectoplasty

INTRODUCTION

 The correction of an anorectal malformation (ARM) is complex and relatively infrequent. Simulation training and subsequent assessment may result in better clinical outcomes. Assessment can be done using a competency assessment tool (CAT). This study aims to develop and validate a CAT for the posterior sagittal anorectoplasty (PSARP) on a simulation model.

MATERIALS AND METHODS

 The CAT-PSARP was developed after consultation with experts in the field. The PSARP was divided into five steps, while tissue and instrument handling were scored separately. Participants of pediatric colorectal hands-on courses in 2019 and 2020 were asked to participate. They performed one PSARP procedure on an ARM simulation model, while being assessed by two objective observers using the CAT-PSARP.

RESULTS

 A total of 82 participants were enrolled. A fair interobserver agreement was found for general skills (intraclass correlation coefficient [ICC] = 0.524, p < 0.001), a good agreement for specific skills (ICC = 0.646, p < 0.001), and overall performance (ICC = 0.669, p < 0.001). The experienced group scored higher on all steps (p < 0.001), except for "anoplasty (p = 0.540)," compared with an inexperienced group.

CONCLUSION

 The CAT-PSARP is a suitable objective assessment tool for the overall performance of the included steps of the PSARP for repair of an ARM on a simulation model.

Resident Skills Assessment in Corneal Suturing: A Comprehensive Review of Currently Proposed Educational Programs and Evaluation Tools

Purpose This study aimed to perform a comprehensive review of publications proposing educational programs for resident skills assessment in corneal suturing.

Methods An extensive online article search in PubMed following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PISMA) reporting guidelines was performed to identify prospective comparative studies or prospective before/after studies published up to March 2021 and reporting the assessment of ophthalmology residents' skills in corneal suturing during dedicated training sessions.

Results Three studies were identified for review. The first reported the efficiency of an electromagnetic tracking system placed on the surgeon's fingers coupled with a computer analysis of movements and time to identify surgeons with different backgrounds in corneal suturing. The second reported the efficiency of the reference system in assessing the improvement of corneal suturing conducted by residents after a training session, with video-based assessment for economy and confidence of movement, limiting tissue damage and precision of operative technique, reviewed by blind assessors. The third proposed an innovative remote corneal suturing training method using Zoom for direct feedback to the residents. The stitch quality was assessed for length, depth, radiality, and tension. The results were similar when compared with a group of residents without feedback.

Conclusion This review underlines the rarity and disparity of available tools for corneal suturing assessment, justifying the need for more complete models to be designed. These should consider body ergonomics and stitch quality and time. Comparative studies involving novices and attendings may provide reliable evaluation of existing gaps and specific metrics to target, helping residents to approach their superiors' experience.

Value of Nontechnical Skills in Minimally Invasive Surgery

Nontechnical skills are of increasing importance in surgery and surgical training. The main studies on its impact on the safety and effectiveness of surgical interventions were mainly published in the first decade of the 2000s. Due to the significant technical complexity and great diversity of instruments for nontechnical skills evaluation, the interest in training and in measuring the impact on surgical safety has relatively decreased. However, the advent of minimally invasive surgery and its peculiar technical characteristics of sophisticated technique and constant innovation through the adoption of new materials and drugs has rekindled interest in this expertise area. In the present review, we have revisited the main instruments available to measure nontechnical skill of surgical teams and analyzed the role of the main competencies on which they are based, such as situational awareness, leadership and communication skills, and the consistency of the intraoperative decision-making process. We conclude that despite the great consensus that exists among all members of the health team on the importance of nontechnical skills for the surgical team in minimally invasive surgery, the reproducible evidence on the subject is scarce and laborious to obtain. To the extent that protecting and expanding nontechnical skills is fundamental to the path toward the high reliability of health institutions, it is possible to anticipate here the role of these institutions as promoters of continuity and new research models in this area of knowledge, especially in minimally invasive surgery, an access route to which more and more patients undergoing surgical treatment in these same institutions are submitted.

Role of Digital Resources in Minimally Invasive Colorectal Surgery Training

Laparoscopic techniques have become the standard for many benign and malignant colorectal pathologies. Recently, the application of robotic-assisted technologies has been, and continues to be, explored. These new technologies require different skill sets and bring novel training challenges, and today's trainees must demonstrate competence in more techniques than ever. Compounding this is the reducing time spent operating in current training programs due to caps on working hours and service pressures. The need for adjunctive training strategies outside the operating room has prompted development of multimedia and digital resources to build the cognitive skills crucial in both nontechnical and technical aspects of surgery. Many are based on principles of cognitive task analysis, breaking down operations, and key decisions into nodal points to be mentally rehearsed. Resources built on this technique have shown improvements in both operative and nonoperative skills, suggesting these resources can advance trainees along the learning curve in minimally invasive surgical techniques. More work to fully elucidate the clinical benefits of such resources is required before their role as a substitute for lost operative training hours can be established. Despite this, alongside other developing technologies such as simulation, they are a promising addition to the armamentarium of the modern-day colorectal trainee.

Nontechnical skills training in cardiothoracic surgery: A pilot study

OBJECTIVE

The importance of nontechnical skills in surgery is widely recognized. We demonstrate the feasibility of administering and assessing the results of a formal Non-Technical Skills in Surgery (NOTSS) curriculum to cardiothoracic surgery residents.

METHODS

Eight cardiothoracic surgery residents participated in the NOTSS curriculum. They were assessed on their cognitive (situation awareness, decision-making) and social (communication and teamwork, leadership) skills based on simulated vignettes. The residents underwent pretraining NOTSS assessments followed by self-administered confidence ratings regarding the 4 skills. Subsequently, a formal NOTSS lecture was delivered and additional readings from the NOTSS textbook was assigned. A month later, the residents returned for post-training NOTSS assessments and self-administered confidence ratings. Changes across days (or within-day before vs after curriculum) were assessed using Wilcoxon signed rank test.

RESULTS

There was a significant improvement in the overall NOTSS assessment score (P = .01) as well as in the individual categories (situation awareness, P = .02; decision-making, P = .02; communication and teamwork, P = .01; leadership, P = .02). There was also an increase in resident self-perception of improvement on the post-training day (P = .01).

CONCLUSIONS

We have developed a simulation-based NOTSS curriculum in cardiothoracic surgery that can be formally integrated into the current residency education. This pilot study indicates the feasibility of reproducible assessments by course educators and self-assessments by participating residents in nontechnical skills competencies.

[Management tasks in surgery: challenges and chances]

Management in surgery is an umbrella term for the activities of a leading surgeon in the fields of strategic planning, organization, coordination, leadership and control. At the same time the management describes the surgeons who carry out the practical administration of a surgical unit. In current times administrative management activities in a surgical unit are taking up an increasing amount of time of a surgeon parallel to the treatment of patients. In the last 17 years this was enhanced by the introduction of the disease-related groups (DRG) system for the classification and billing of surgical treatment within the German hospital healthcare system, which further enhanced the economic pressure on hospitals and surgical units. To enable surgeons to carry out a sustainable management of a surgical unit they need to acquire a variety of management qualities parallel to the advanced training as a specialist and the surgical training.

The Trainee's Role in Patient Safety: Training Residents and Medical Students in Surgical Patient Safety

"The focus on patient safety offers a new framework not only for delivering health care but also for training physicians. Medical school and surgical graduate medical education must transition to a more holistic approach by teaching technical and nontechnical skills. Formalized safety curricula can be developed by adopting recommended guidelines and content from national and international organizations, existing validated practices of training programs, frequent simulation exercises, and objective evaluation tools."

Surgical Competencies Required in Newly Commencing Colorectal Surgeons: an Educational and Training Spectrum

PURPOSE

Surgical training models have changed from master-apprentice to competency-based training. We aimed to determine the relative importance and peak periods of acquiring these competencies in newly commencing colorectal surgeons.

METHODS

A mailed questionnaire to all current Colorectal Surgical Society of Australia and New Zealand (CSSANZ) members was conducted between October and December 2016 assessing the relative importance of each competency and the period or activity of learning or training contributing most to achieving that competency.

RESULTS

The response rate was 43% (90/208) with 87% (n = 75) agreed or strongly agreed to the relevance and applicability of the nine RACS competencies. Competencies varied in perceived importance (strongly agreed: judgment-clinical decision-making (JU) 63%, collaboration/teamwork (CT) 53%, technical expertise (TE) 47%, communication (CO) 44%, medical expertise (ME) 34%, scholarship/teaching (ST) 33%, professionalism (PR) 33%/ethics (ET) 24%, health advocacy (HA) 18%, management (MX) 13%/leadership (LE) 17%), and the peak period for acquiring them (registrar: CO 39%, ST 30%; fellow: TE 62%, CT 44%, ME 40%, JU 38%; consultant: MX/LE 52%, HA 48%, PR/ET 33%).

CONCLUSION

Surgical competencies for colorectal surgeons are accumulated and acquired at varying degrees and periods across a spectrum of continuing registrar, fellow, and consultant education and training. These findings serve as a baseline for further refinement of current and continuing educational and training programs.

Towards Development of an Open Surgery Competency Assessment for Residents (OSCAR) Tool - A Systematic Review of the Literature and Delphi Consensus

OBJECTIVE

Competency-based education has mandated accurate intra-operative assessment tools. We aimed to define consensus-based open surgical skills perceived by experts as critical for assessment.

DESIGN

A mixed-method design was employed: systematic review and e-Delphi methodology.

SETTING

The study was performed at McGill University-affiliated large tertiary academic centers in Montreal, Quebec, Canada.

PARTICIPANTS

Per PRISMA guidelines, a peer-reviewed search strategy was employed. Studies published in English and those describing technical skill assessment of open abdominal surgery were included; subspecialty-specific skills, conference abstracts, academic memoirs were excluded. Most-cited skills were subjected to e-Delphi methodology to identify those deemed essential by experts, based a 3-point Likert scale. Eighteen McGill University-affiliated general surgeons, representing a variety of subspecialties of General Surgery, were invited to answer the questionnaire.

RESULTS

Around 120 of 4285 references were retained for analysis. The 12 most cited skills included suturing, tissue and instrument handling, movement economy, instrument knowledge, knot tying, flow, knowledge of procedure, completion time, dissection technique, knowledge of anatomy and sterile technique; 6 of these achieved high or perfect scores and agreement after 2 rounds of survey: suturing, sterile technique, knot tying, knowledge of anatomy, knowledge of procedure, and tissue handling. Median standard deviation decreased (0.495 to 0.450) from first to second round, indicating improvement in consensus.

CONCLUSION

These results will help develop and validate the OSCAR (objective structured clinical assessment rubric) assessment tool for immediate intra-operative feedback of open technical skills for surgical trainees.

Non-technical skills assessments in undergraduate medical education: A focused BEME systematic review: BEME Guide No. 54

Consensus on how to assess non-technical skills is lacking. This systematic review aimed to evaluate the evidence regarding non-technical skills assessments in undergraduate medical education, to describe the tools used, learning outcomes and the validity, reliability and psychometrics of the instruments. A standardized search of online databases was conducted and consensus reached on included studies. Data extraction, quality assessment, and content analysis were conducted per Best Evidence in Medical Education guidelines. Nine papers met the inclusion criteria. Assessment methods broadly fell into three categories: simulated clinical scenarios, objective structured clinical examinations, and questionnaires or written assessments. Tools to assess non-technical skills were often developed locally, without reference to conceptual frameworks. Consequently, the tools were rarely validated, limiting dissemination and replication. There were clear themes in content and broad categories in methods of assessments employed. The quality of this evidence was poor due to lack of theoretical underpinning, with most assessments not part of normal process, but rather produced as a specific outcome measure for a teaching-based study. While the current literature forms a good starting position for educators developing materials, there is a need for future work to address these weaknesses as such tools are required across health education.

The Alternative Surgical Curriculum

Surgical training has focused on the development of technical competency. Interpersonal and cognitive skills are essential to working as an interdisciplinary team, which translates into safety for the patient and well-being for the surgeon and colleagues. This article offers an "alternative" surgical curriculum topic list to augment the technical skill sets traditionally taught to trainees.

Identification of Warning Signs During Selection of Surgical Trainees

OBJECTIVE

The aim was to document empirical observations about antecedents to and practices of unsuitable behaviours amongst surgical trainees and develop an interview guide that could be used for the selection process.

DESIGN

A mixed methods design was adopted combining a survey distributed to senior surgeons and heads of departments, followed by semi-structured interviews with experienced surgeons.

SETTING

All surgical departments and hospitals in The South Swedish Health Care Region.

PARTICIPANTS

The survey was completed by 54 of 83 eligible surgeons above 50years of age, and 4 of 7 heads of surgical departments. Semi-structured interviews with 13 surgeons representing local, regional, and university hospitals from the same cohort.

RESULTS

Forty-six (85%) surgeons and four of seven heads of departments responded that they had come across surgical trainees deemed unsuitable to train and work as a surgeon. All heads of department and 31 of 54 of the surgeons believed tendencies towards unsuitability are evident early during training. From the survey, 107 statements described reasons for finding a trainee unsuitable. Qualitative analysis of the interviews and free-text answers of the survey led to identification of 11 problem domains with associated "warning signs". An interview guide to help detect unsuitability tendencies in candidates during selection procedures was constructed.

CONCLUSIONS

Experienced surgeons have quite consistent views on what makes a person unsuitable as a surgeon. Their views have been systematized into 11 problem domains, and a set of 'warning signs' for unsuitable behaviours and traits has been developed. Early detection of these signs and traits is important for the individual, the work environment, and patient safety. A recommendation for a minimum framework for selection including the constructed interview guide is presented.

Performance anxiety among surgeons

In the first of our psychology of surgery series, we look at surgeons’ performance anxiety from a psychological perspective.

Toward a Knowledge-Driven Context-Aware System for Surgical Assistance

Complex surgeries complications are increasing, thus making an efficient surgical assistance is a real need. In this work, an ontology-based context-aware system was developed for surgical training/assistance during Thoracentesis by using image processing and semantic technologies. We evaluated the Thoracentesis ontology and implemented a paradigmatic test scenario to check the efficacy of the system by recognizing contextual information, e.g. the presence of surgical instruments on the table. The framework was able to retrieve contextual information about current surgical activity along with information on the need or presence of a surgical instrument.

Modeling Surgical Resident Performance

The goals of this work were to evaluate the methods used for assessing surgical residents and to model surgical resident performance. Currently, residents are evaluated by the attending surgeons using a one-page paper evaluation form after each rotation in a particular department. These subjective questionnaires require the evaluators to rate residents in competency areas that are thought to define a successful surgeon. An electronic database was created from resident performance records collected over the past 33 years from the Department of Surgery at the Tufts-NEMC. A usability study examined the effectiveness of the design of the evaluation form and the competency measures. Analysis showed nine changes in format between 1972 and 2005, varying in the competencies rated and rating scales used. Regression analysis was used to model the performance of surgical residents. Results showed that judgment ( p < 0.0001), initiative ( p < 0.0001), and reaction to stress ( p = 0.0206) were significant predictors of a successful outcome. This model may be used to predict the success of new residents and possibly target weaknesses in the surgical education curriculum.

Teamwork Assessment Tools in Modern Surgical Practice: A Systematic Review

Introduction. Deficiencies in teamwork skills have been shown to contribute to the occurrence of adverse events during surgery. Consequently, several teamwork assessment tools have been developed to evaluate trainee nontechnical performance. This paper aims to provide an overview of these instruments and review the validity of each tool. Furthermore, the present paper aims to review the deficiencies surrounding training and propose several recommendations to address these issues. Methods. A systematic literature search was conducted to identify teamwork assessment tools using MEDLINE (1946 to August 2015), EMBASE (1974 to August 2015), and PsycINFO (1806 to August 2015) databases. Results. Eight assessment tools which encompass aspects of teamwork were identified. The Nontechnical Skills for Surgeons (NOTSS) assessment was found to possess the highest level of validity from a variety of sources; reliability and acceptability have also been established for this tool. Conclusions. Deficits in current surgical training pathways have prompted several recommendations to meet the evolving requirements of surgeons. Recommendations from the current paper include integration of teamwork training and assessment into medical school curricula, standardised formal training of assessors to ensure accurate evaluation of nontechnical skill acquisition, and integration of concurrent technical and nontechnical skills training throughout training.

The Relationship Between Technical And Nontechnical Skills Within A Simulation-Based Ureteroscopy Training Environment

OBJECTIVE

Little integration of technical and nontechnical skills (e.g., situational awareness, communication, decision making, teamwork, and leadership) teaching exists within surgery. We therefore aimed to (1) evaluate the relationship between these 2 skill sets within a simulation-based environment and (2) assess if certain nontechnical skill components are of particular relevance to technical performance.

DESIGN

A prospective analysis of data acquired from a comparative study of simulation vs nonsimulation training was conducted. Half of the participants underwent training of technical and nontechnical skills within ureteroscopy, with the remaining half undergoing no training. All were assessed within a full immersion environment against both technical (time to completion, Objective Structured Assessment of Technical Skills, and task-specific checklist scores) and nontechnical parameters (Nontechnical Skills for Surgeons [NOTSS] rating scale). The data of whole and individual cohorts were analyzed using Pearson correlation coefficient.

SETTING

The trial took place within the Simulation and Interactive Learning Centre at Guy's Hospital, London, UK.

PARTICIPANTS

In total, 32 novice participants with no prior practical ureteroscopy experience were included within the data analysis.

RESULTS

A correlation was found within all outcome measures analyzed. For the whole cohort, a strong negative correlation was found between time to completion and NOTSS scores (r = -0.75, p < 0.001), with strong positive correlations identified when NOTSS scores were compared with Objective Structured Assessment of Technical Skills (r = 0.89, p < 0.001) and task-specific checklist scores (r = 0.91, p < 0.001). Similar results were observed when each cohort was analyzed separately. Finally, all individual nontechnical skill components demonstrated a strong correlation with all technical skill parameters, regardless of training.

CONCLUSIONS

A strong correlation between technical and nontechnical performance exists, which was demonstrated to be irrespective of training received. This may suggest an inherent link between skill sets. Furthermore, all nontechnical skill sets are important in technical performance. This supports the notion that both of these skills should be trained and assessed together within 1 curriculum.

Reflections: Surgical Education-the Times they are a-Changin': Lessons Learned from the 3rd MAYMET-ESO Joint Meeting

Technical skills are not sufficient for successful surgical care. Non-technical skills such as team work, decision-making in cancer treatment, communication with the patient, ethical challenges, situation awareness, and communication in the operating room are mandatory for favorable outcomes. Although formally taught in other high-demand disciplines, such skills were traditionally rarely discussed in surgical oncology. The 3rd MAYMET-ESO Joint Meeting "Professionalism for Breast Surgeons" held in Istanbul, Turkey, 5 October 2013 was dedicated to the development of non-technical skills in the everyday activity of breast surgeons. We briefly discuss information from this very interesting and inspiring educational event and how it relates to more recent changes in surgical oncology education.

Making meaning from sensory cues: a qualitative investigation of postgraduate learning in the operating room

PURPOSE

The authors aimed to map and explicate what surgeons perceive they learn in the operating room.

METHOD

The researchers used a grounded theory method in which data were iteratively collected through semistructured one-to-one interviews in 2010 and 2011 at four participating hospital sites. A four-person data analysis team from differing academic backgrounds qualitatively analyzed the content of the transcripts employing an immersion/crystallization approach.

RESULTS

Participants were 22 UK surgeons, some of whom were in training at the time of the study and some of whom were attending surgeons. Major themes of learning in the operating room were perceived to be factual knowledge, motor skills, sensory semiosis, adaptive strategies, team working and management, and attitudes and behaviors. The analysis team classified 277 data points (short paragraphs or groups of sentences conveying meaning) under these major themes and subthemes. A key component of learning in the operating room that emerged from these data was sensory semiosis, defined as learning to make sense of visual and haptic cues.

CONCLUSIONS

Although the authors found that learning in the operating room occurred across a wide range of domains, sensory semiosis was found to be an important theme that has not previously been fully acknowledged or discussed in the surgical literature. The discussion draws on the wider literature from the social sciences and cognitive psychology literature to examine how professionals learn to make meaning from "signs" making parallels with other medical specialties.

Multimedia educational tools for cognitive surgical skill acquisition in open and laparoscopic colorectal surgery: a randomized controlled trial

AIM

Conventional teaching in surgical training programmes is constrained by time and cost, and has room for improvement. This study aimed to determine the effectiveness of a multimedia educational tool developed for an index colorectal surgical procedure (anterior resection) in teaching and assessment of cognitive skills and to evaluate its acceptability amongst general surgical trainees.

METHOD

Multimedia educational tools in open and laparoscopic anterior resection were developed by filming multiple operations which were edited into procedural steps and substeps and then integrated onto interactive navigational platforms using Adobe® Flash® Professional CS5 10.1. A randomized controlled trial was conducted on general surgical trainees to evaluate the effectiveness of online multimedia in comparison with conventional 'study day' teaching for the acquisition of cognitive skills. All trainees were assessed before and after the study period. Trainees in the multimedia group evaluated the tools by completing a survey.

RESULTS

Fifty-nine trainees were randomized but 27% dropped out, leaving 43 trainees randomized to the multimedia group (n = 25) and study day group (n = 18) who were available for analysis. Posttest scores improved significantly in both groups (P < 0.01). The change in scores (mean ± SD) in the multimedia group was not significantly different from the study day group (6.02 ± 5.12 and 5.31 ± 3.42, respectively; P = 0.61). Twenty-five trainees completed the evaluation survey and experienced an improvement in their decision making (67%) and in factual and anatomical knowledge (88%); 96% agreed that the multimedia tool was a useful additional educational resource.

CONCLUSION

Multimedia tools are effective for the acquisition of cognitive skills in colorectal surgery and are well accepted as an educational resource.

The Stanford Microsurgery and Resident Training (SMaRT) Scale: validation of an on-line global rating scale for technical assessment

INTRODUCTION

We previously reported results of our on-line microsurgery training program, showing that residents who had access to our website significantly improved their cognitive and technical skills. In this study, we report an objective means for expert evaluators to reliably rate trainees' technical skills under the microscope, with the use of our novel global rating scale.

METHODS

"Microsurgery Essentials" (http://smartmicrosurgery.com) is our on-line training curriculum. Residents were randomly divided into 2 groups: 1 group reviewed this online resource and the other did not. Pre- and post-tests consisted of videotaped microsurgical sessions in which the trainee performed "microsurgery" on 3 different models: latex glove, penrose drain, and the dorsal vessel of a chicken foot. The SMaRT (Stanford Microsurgery and Resident Training) scale, consisting of 9 categories graded on a 5-point Likert scale, was used to assess the trainees. Results were analyzed with ANOVA and Student t test, with P less than 0.05 indicating statistical significance.

RESULTS

Seventeen residents participated in the study. The SMaRT scale adequately differentiated the performance of more experienced senior residents (PGY-4 to PGY-6, total average score=3.43) from less experienced junior residents (PGY-1 to PGY-3, total average score=2.10, P<0.0001). Residents who viewed themselves as being confident received a higher score on the SMaRT scale (average score 3.5), compared to residents who were not as confident (average score 2.1) (P<0.001). There were no significant differences in scoring among all 3 evaluators (P>0.05). Additionally, junior residents who had access to our website showed a significant increase in their graded technical performance by 0.7 points when compared to residents who did not have access to the website who showed an improvement of only 0.2 points (P=0.01).

CONCLUSIONS

Our SMaRT scale is valid and reliable in assessing the microsurgical skills of residents and other trainees. Current trainees are more likely to use self-directed on-line education because of its easy accessibility and interactive format. Our global rating scale can help ensure residents are achieving appropriate technical milestones.

Development and face validation of strategies for improving consultation skills

While formative workplace based assessment can improve learners' skills, it often does not because the procedures used do not facilitate feedback which is sufficiently specific to scaffold improvement. Provision of pre-formulated strategies to address predicted learning needs has potential to improve the quality and automate the provision of written feedback. To systematically develop, validate and maximise the utility of a comprehensive list of strategies for improvement of consultation skills through a process involving both medical students and their clinical primary and secondary care tutors. Modified Delphi study with tutors, modified nominal group study with students with moderation of outputs by consensus round table discussion by the authors. 35 hospital and 21 GP tutors participated in the Delphi study and contributed 153 new or modified strategies. After review of these and the 205 original strategies, 265 strategies entered the nominal group study to which 46 year four and five students contributed, resulting in the final list of 249 validated strategies. We have developed a valid and comprehensive set of strategies which are considered useful by medical students. This list can be immediately applied by any school which uses the Calgary Cambridge Framework to inform the content of formative feedback on consultation skills. We consider that the list could also be mapped to alternative skills frameworks and so be utilised by schools which do not use the Calgary Cambridge Framework.

Assessment of surgery residents' operative skills in the operating theater using a modified Objective Structured Assessment of Technical Skills (OSATS): a prospective multicenter study

BACKGROUND

With the implementation of competency-based curricula, Objective Structured Assessment of Technical Skills (OSATS) increasingly is being used for the assessment of operative skills. Although evidence for its usefulness has been demonstrated in experimental study designs, data supporting OSATS application in the operating room are limited. This study evaluates the validity and reliability of the OSATS instrument to assess the operative skills of surgery residents in the operating theater.

METHODS

Twenty-four residents were recruited from seven hospitals within a general surgical training region and classified equally into three groups according to postgraduate training year (PGY). Each resident had to perform five different types of operations. Surgical performance was measured using a modified OSATS consisting of three scales: Global Rating Scale, Overall Performance Scale, and Alphabetic Summary Scale. Validity and reliability metrics included construct validity (Kruskal-Wallis test) and internal consistency reliability (Cronbach's α coefficient). Spearman's correlation coefficients were calculated to determine correlations between the different scales.

RESULTS

Eighteen residents (PGY 1-2 [n = 7]; PGY 3-4 [n = 8]; PGY 5-6 [n = 3]) performed 249 operations. Comparisons of the performance scores revealed that evidence for construct validity depended on the difficulty level of the selected procedures. For individual operations, internal consistency reliability of the Global Rating Scale ranged from 0.93 to 0.95. Scores on the different scales correlated strongly (r = 0.62-0.76, P < .001).

CONCLUSION

Assessment of operative skills in the operating theater using this modified OSATS instrument has the potential to establish learning curves, allowing adequate monitoring of residents' progress in achieving operative competence. The Alphabetic Summary Scale seems to be of additional value. Use of the Overall Performance Scale should be reconsidered.

Observation of behavioural markers of non-technical skills in the operating room and their relationship to intra-operative incidents

BACKGROUND

The importance of non-technical skills in improving surgical safety and performance is now well recognised. Better understanding is needed of the impact that non-technical skills of the multi-disciplinary theatre team have on intra-operative incidents in the operating room (OR) using structured theatre-based assessment. The interaction of non-technical skills that influence surgical safety of the OR team will be explored and made more transparent.

METHODS

Between May-August 2013, a range of procedures in general and vascular surgery in the Royal Infirmary of Edinburgh were performed. Non-technical skills behavioural markers and associated intra-operative incidents were recorded using established behavioural marking systems (NOTSS, ANTS and SPLINTS). Adherence to the surgical safety checklist was also observed.

RESULTS

A total of 51 procedures were observed, with 90 recorded incidents - 57 of which were considered avoidable. Poor situational awareness was a common area for surgeons and anaesthetists leading to most intra-operative incidents. Poor communication and teamwork across the whole OR team had a generally large impact on intra-operative incidents. Leadership was shown to be an essential set of skills for the surgeons as demonstrated by the high correlation of poor leadership with intra-operative incidents. Team-working and management skills appeared to be especially important for anaesthetists in the recovery from an intra-operative incident.

CONCLUSION

A significant number of avoidable incidents occur during operative procedures. These can all be linked to failures in non-technical skills. Better training of both individual and team in non-technical skills is needed in order to improve patient safety in the operating room.

S/he who can, does and teaches. S/he who cannot, doesn't

BACKGROUND

The saying, "[h]e who can, does. He who cannot, teaches." suggests that those who have the skills to perform do so, whereas those who do not have those skills become teachers. We hypothesize that this saying as it relates to general surgery residents is not true.

METHODS

This was a retrospective study of general surgery chief residents from 2009 to 2013. Technical ability was assessed with the Fundamentals of Laparoscopic Surgery examination performance. Teaching ability was assessed with medical student evaluations on a 9-point Likert scale as well as with receipt of teaching awards: The Arnold P. Gold Teaching Award, the surgical teaching award given by each graduating class of the medical school, and resident induction into Alpha Omega Alpha. Mann-Whitney U tests were performed between resident groups based on teaching award status and Fundamentals of Laparoscopic Surgery examination outcomes, using an α = 0.05.

RESULTS

For 32 chief residents (7 female), the median score on the Manual Skills Section was 531 (interquartile range [IQR] [478-563]). On the Cognitive Section, the cohort of residents who won each award outperformed the residents who did not win that specific award (all p < 0.05). On the Manual Skills Section, residents who received any teaching award/the Arnold P. Gold Teaching Award (n = 7) performed similar to residents who did not receive this award (n = 25) (584 [IQR {491-620}] vs. 527 [IQR {482-537}]) (p = 0.09). Residents who received the surgical teaching award from the medical school (n = 3) performed similar to residents who did not receive this award (n = 29) (608 [IQR {474-637}] vs. 527 [IQR {482-555}]) (p = 0.23). Eligible residents who were inducted into Alpha Omega Alpha (n = 4) outperformed residents who were not inducted (n = 18) (596 [IQR {564-637}] vs. 527 [IQR {446-551}]) (p = 0.01).

CONCLUSIONS

There is some evidence that superior resident teachers have greater content knowledge and a higher degree of laparoscopic skills.

Surgeons' attitude toward a competency-based training and assessment program: results of a multicenter survey

BACKGROUND

Currently, most surgical training programs are focused on the development and evaluation of professional competencies. Also in the Netherlands, competency-based training and assessment programs were introduced to restructure postgraduate medical training. The current surgical residency program is based on the Canadian Medical Education Directives for Specialists (CanMEDS) competencies and uses assessment tools to evaluate residents' competence progression. In this study, we examined the attitude of surgical residents and attending surgeons toward a competency-based training and assessment program used to restructure general surgical training in the Netherlands in 2009.

METHODS

In 2011, all residents (n = 51) and attending surgeons (n = 108) in 1 training region, consisting of 7 hospitals, were surveyed. Participants were asked to rate the importance of the CanMEDS competencies and the suitability of the adopted assessment tools. Items were rated on a 5-point Likert scale and considered relevant when at least 80% of the respondents rated an item with a score of 4 or 5 (indicating a positive attitude). Reliability was evaluated by calculating the Cronbach's α, and the Mann-Whitney test was applied to assess differences between groups.

RESULTS

The response rate was 88% (n = 140). The CanMEDS framework demonstrated good reliability (Cronbach's α = 0.87). However, the importance of the competencies 'Manager' (78%) and 'Health Advocate' (70%) was undervalued. The assessment tools failed to achieve an acceptable reliability (Cronbach's α = 0.55), and individual tools were predominantly considered unsuitable for assessment. Exceptions were the tools 'in-training evaluation report' (91%) and 'objective structured assessment of technical skill' (82%). No significant differences were found between the residents and the attending surgeons.

CONCLUSION

This study has demonstrated that, 2 years after the reform of the general surgical residency program, residents and attending surgeons in a large training region in the Netherlands do not acknowledge the importance of all CanMEDS competencies and consider the assessment tools generally unsuitable for competence evaluation.

The traveling salesman problem in surgery: economy of motion for the FLS Peg Transfer task

BACKGROUND

In the Peg Transfer task in the Fundamentals of Laparoscopic Surgery (FLS) curriculum, six peg objects are sequentially transferred in a bimanual fashion using laparoscopic instruments across a pegboard and back. There are over 268 trillion ways of completing this task. In the setting of many possibilities, the traveling salesman problem is one where the objective is to solve for the shortest distance traveled through a fixed number of points. The goal of this study is to apply the traveling salesman problem to find the shortest two-dimensional path length for this task.

METHODS

A database platform was used with permutation application output to generate all of the single-direction solutions of the FLS Peg Transfer task. A brute-force search was performed using nested Boolean operators and database equations to calculate the overall two-dimensional distances for the efficient and inefficient solutions. The solutions were found by evaluating peg object transfer distances and distances between transfers for the nondominant and dominant hands.

RESULTS

For the 518,400 unique single-direction permutations, the mean total two-dimensional peg object travel distance was 33.3 ± 1.4 cm. The range in distances was from 30.3 to 36.5 cm. There were 1,440 (0.28 %) of 518,400 efficient solutions with the minimized peg object travel distance of 30.3 cm. There were 8 (0.0015 %) of 518,400 solutions in the final solution set that minimized the distance of peg object transfer and minimized the distance traveled between peg transfers. Peg objects moved 12.7 cm (17.4 %) less in the efficient solutions compared to the inefficient solutions.

CONCLUSIONS

The traveling salesman problem can be applied to find efficient solutions for surgical tasks. The eight solutions to the FLS Peg Transfer task are important for any examinee taking the FLS curriculum and for certification by the American Board of Surgery.

Systematic review of predictors of surgical performance

Background

Selection criteria for surgical training are not scientifically proven. There is a need to define which attributes predict future surgical performance. The aim of this study was to examine the predictive value of specific attributes that impact on surgical performance.

Methods

All studies assessing the predictive power of specified attributes with regard to outcome measures of surgical performance in MEDLINE, Embase, the Cochrane Central Register of Controlled Trials and Educational Resources Information Centre databases, and bibliographies of selected articles from 1950 to November 2010 were considered for inclusion by two independent reviewers. Information on study identifiers, participant characteristics, predictors assessed, evaluation methods for predictors, outcome measures, results and statistical analysis was collected. Quality assessment was carried out using the Hayden criteria.

Results

Visual–spatial perception correlated with both subjective and objective assessments of surgical performance, including rate of skill acquisition. Visual–spatial perception did not correlate with operative ability in experts, although it did with operative ability at the end of a training programme. Psychomotor aptitude, assessed collectively, correlated with rate of skill acquisition. Academic achievement predicted completion of a training programme and passing end-of-training examinations, but did not predict clinical performance during the training programme.

Conclusion

Intermediate- and high-level visual–spatial perception, as well as psychomotor aptitude, can be used as criteria for assessing candidates for surgical training. Academic achievement is an effective predictor of successful completion of training programmes and should continue to form part of the assessment of surgical candidates.

Interprofessional non-technical skills for surgeons in disaster response: a qualitative study of the Australian perspective

Interprofessional non-technical skills for surgeons in disaster response have not yet been developed. The aims of this study were to identify the non-technical skills required of surgeons in disaster response and training for disaster response and to explore the barriers and facilitators to interprofessional practice in surgical teams responding to disasters. Twenty health professionals, with prior experience in natural disaster response or education, participated in semi-structured in-depth interviews. A qualitative matrix analysis design was used to thematically analyze the data. Non-technical skills for surgeons in disaster response identified in this study included skills for austere environments, cognitive strategies and interprofessional skills. Skills for austere environments were physical self-care including survival skills, psychological self-care, flexibility, adaptability, innovation and improvisation. Cognitive strategies identified in this study were "big picture" thinking, situational awareness, critical thinking, problem solving and creativity. Interprofessional attributes include communication, team-player, sense of humor, cultural competency and conflict resolution skills. "Interprofessionalism" in disaster teams also emerged as a key factor in this study and incorporated elements of effective teamwork, clear leadership, role adjustment and conflict resolution. The majority of participants held the belief that surgeons needed training in non-technical skills in order to achieve best practice in disaster response. Surgeons considerring becoming involved in disaster management should be trained in these skills, and these skills should be incorporated into disaster preparation courses with an interprofessional focus.

Surgeons' non-technical skills

The importance of non-technical skills to surgical performance is gaining wide acceptance. This article discusses the core cognitive and social skills categories thought to underpin medical knowledge and surgical expertise, and describes the rise of non-technical skill models of assessment in surgery. Behavior rating systems such as NOTSS (Non-Technical Skills for Surgeons) have been developed to support education and assessment in this regard. We now understand more about these critical skills and how they impact surgery. The challenge in the future is to incorporate them into undergraduate teaching, postgraduate training, workplace assessment, and perhaps even selection.

Establishing hand preference: why does it matter?

Hand preference has been associated with psychological and physical well-being, risk of injury, pathological irregularities, longevity, and cognitive function. To determine hand preference, individuals are often asked what hand they use to write with, or what hand is used more frequently in activities of daily living. However, relying only on one source of information may be misleading, given the strong evidence to support a disassociation between self-reported hand preference and outcomes of hand performance assessments. This brief communication is intended to highlight the various methods used to determine hand preference, to discuss the relationship between hand preference inventories and performance measures and to present some recent findings associated with hand preference and musculoskeletal disorders.

Observational tools for assessment of procedural skills: a systematic review

BACKGROUND

Assessment by direct observation of procedural skills is an important source of constructive feedback. The aim of this study was to identify observational tools for technical skill assessment, to assess characteristics of these tools, and to assess their usefulness for assessment.

METHODS

Included studies reported tools for observational assessment of technical skills. A total of 106 articles were included.

RESULTS

Three main categories included global assessment scales evaluating generic skills (n = 29), task-specific methods assessing procedure-specific skills (n = 30), and combinations of tools evaluating both generic and task-specific skills (n = 47). In most studies, content validity was not evaluated using an accepted scientific method. All tools were assessed for inter-rater reliability and construct validity. Data on feasibility, acceptability, and educational impact were sparse.

CONCLUSIONS

There is evidence of validity and reliability for observational assessment tools at the trainee level. In most studies a comprehensive analysis of the tools was not achieved. Evaluation of technical skill using current observational assessment tools is not reliable and valid at the specialist level. Future research needs to focus on further systematic tool development and analysis, especially at the specialist level.

Measuring processes of care in general surgery: assessment of technical and nontechnical skills

Measuring the quality of health care is becoming increasingly important. Quality is often conceptualized as 3 dimensions of care: structures, processes, and outcomes. Unfortunately, there is little consensus about what should be measured--and how it should be measured--when it comes to measuring processes of care related to the conduct of surgical procedures. This article reviews recent advances in surgical quality of care measurement with particular emphasis on processes of care, and evaluates existing measures of technical and nontechnical surgical skills as measures of quality of care in surgery.