Randomized controlled trial on the effect of coaching in simulated laparoscopic training

Constructing psychometric measurement of a past supervised experience scale with educational roles to investigate the relationship between being instructed, being supported, and being supervised for attending physicians in teaching hospitals - a questionnaire survey

PURPOSE

Although school-based planned education has steadily been introduced, master-apprentice learning is still an indispensable part of medical education. All medical clinical teachers begin their careers as medical students, often without knowing exactly how they will learn to teach. Kilminster and Jolly identified three primary functions of clinical supervisors including clinical teaching, support and guidance, and work supervision. Therefore, we designed a study using questionnaires to assess the three factors of clinical educators for past supervised experiences, including 'being instructed,' 'being supported,' and 'being supervised,' based on Kilminster and Jolly's illustration in 2000, to see the relationship between the three factors mentioned above.

MATERIALS AND METHODS

The study started with a literature review to construct the essential items regarding past supervised experiences of physicians as clinical teachers. We invited 10 experts from fields including medical education and experienced clinical teachers to assess the content validity. One hundred physicians in teaching hospitals were sampled for the preliminary test. Another 364 physicians in teaching hospitals were sampled for the formal study of confirmatory factor analysis and pathway analysis.

RESULTS

The" Past Supervised Experiences with Educational Roles Scale" showed satisfying reliability with all Cronbach's α values exceeding .80, and three factors from supervised experiences were identified, including 'being supported,' 'being instructed,' and 'being supervised.' In our model, the 'being supported' experience could positively affect 'being supervised' with significance, directly and indirectly, by being instructed.

CONCLUSIONS

Our study developed a validated instrument that allows investigation of the formation of better-supervised experiences from current physicians. Our findings inspired us to focus more on supportive coaching in teaching and supervising medical trainees. Our study indicated that faculty development for skills of supporting students is crucial to effective clinical teaching and supervision.

Video-Based Performance Analysis in Pituitary Surgery - Part 2: Artificial Intelligence Assisted Surgical Coaching

BACKGROUND

Superior surgical skill improves surgical outcomes in endoscopic pituitary adenoma surgery. Video-based coaching programs, pioneered in professional sports, have shown promise in surgical training. In this study, we developed and assessed a video-based coaching program using artificial intelligence (AI) assistance.

METHODS

An AI-assisted video-based surgical coaching was implemented over 6 months with the pituitary surgery team. The program consisted of 1) monthly random video analysis and review; and 2) quarterly 2-hour educational meetings discussing these videos and learning points. Each video was annotated for surgical phases and steps using AI, which improved video interactivity and allowed the calculation of quantitative metrics. Primary outcomes were program feasibility, acceptability, and appropriateness. Surgical performance (via modified Objective Structured Assessment of Technical Skills) and early surgical outcomes were recorded for every case during the 6-month coaching period, and a preceding 6-month control period. Beta and logistic regression were used to assess the change in modified Objective Structured Assessment of Technical Skills scores and surgical outcomes after the coaching program implementation.

RESULTS

All participants highly rated the program's feasibility, acceptability, and appropriateness. During the coaching program, 63 endoscopic pituitary adenoma cases were included, with 41 in the control group. Surgical performance across all operative phases improved during the coaching period (P < 0.001), with a reduction in new postoperative anterior pituitary hormone deficit (P = 0.01).

CONCLUSIONS

We have developed a novel AI-assisted video surgical coaching program for endoscopic pituitary adenoma surgery - demonstrating its viability and impact on surgical performance. Early results also suggest improvement in patient outcomes. Future studies should be multicenter and longer term.

Educational Effectiveness of Telementoring as a Continuing Professional Development Intervention for Surgeons in Practice: A Systematic Review

Objective

We performed a systematic review to determine the educational effectiveness of telementoring as a continuing professional development (CPD) intervention for surgeons in practice.

Background

Surgeons can mentor their peers in remote locations using videoconferencing communication, referred to as telementoring.

Methods

We searched MEDLINE and EMBASE and included studies assessing the educational effectiveness of telementoring interventions used by surgeons in practice. We excluded studies involving only trainees and those not evaluating educational effectiveness. Two reviewers independently screened, extracted data, and assessed study quality using the Medical Education Research Study Quality Instrument (MERSQI; maximum score 18). Educational outcomes were categorized using Moore's Outcomes Framework.

Results

We retrieved a total of 1351 records, and 252 studies were selected for full-text review. Twenty-eight studies were included with 1 randomized controlled trial, 19 cohort studies, 5 qualitative studies, and 3 case studies, totaling 178 surgeons and 499 cases. The average MERSQI score was 10.21 ± 2.2 out of 18. Educational outcomes included surgeons' satisfaction with telementoring interventions (Moore's Level 2) in 12 studies, improvement in surgeons' procedural knowledge (Level 3b) in 3 studies, improvements in surgeons' procedural competence in an educational setting (Level 4) in 4 studies, performance in a workplace-based setting (Level 5) in 23 studies, and patient outcomes (Level 6) in 3 studies. No studies reported community health outcomes (Level 7).

Conclusions

Moderate-level evidence demonstrates the use of telementoring as effective in changing surgeons' knowledge and competence in both educational and workplace-based settings. Its use is also associated with changes in patient outcomes.

Coaching with audiovisual technology in acute-care hospital settings: systematic review

BACKGROUND

Surgical coaching programmes are a means of improving surgeon performance. Embedded audiovisual technology has the potential to further enhance participant benefit and scalability of coaching. The objective of this systematic review was to evaluate how audiovisual technology has augmented coaching in the acute-care hospital setting and to characterize its impact on outcomes.

METHODS

A systematic review was conducted, searching PubMed, Ovid MEDLINE, Embase, PsycInfo, and CINAHL databases using PRISMA. Eligible studies described a coaching programme that utilized audiovisual technology, involved at least one coach-coachee interaction, and included healthcare professionals from the acute-care hospital environment. The risk of bias 2 tool and grading of recommendations, assessment, development, and evaluations (GRADE) framework were used to evaluate studies. Synthesis without meta-analysis was performed, creating harvest plots of three coaching outcomes: technical skills, self-assessment/feedback, and non-technical skills.

RESULTS

Of 10 458 abstracts screened, 135 full texts were reviewed, and 21 studies identified for inclusion. Seventeen studies were conducted within surgical specialties and six classes of audiovisual technology were utilized. An overall positive direction of effect was demonstrated for studies measuring improvement of either technical skills or non-technical skills. Direction of effect for self-assessment/feedback was weakly positive.

CONCLUSION

Audiovisual technology has been used successfully in coaching programmes within acute-care hospital settings to facilitate or assess coaching, with a positive impact on outcome measures. Future studies may address the additive benefits of video over in-person observation and enhance the certainty of evidence that coaching impacts on surgeon performance, surgeon well-being, and patient outcomes.

Residents as Medical Coaches

OBJECTIVES

With the recent implementation of Competency-based Medical Education (CBME) and emphasis on direct observation of learners, there is an increased interest in the concept of clinical coaching. While there is considerable literature on the role of attending physicians as coaches, little data is available on the role of residents as coaches, and residents' perceptions about effective coaching. We aimed to identify distinct characteristics of residents' coaching, to examine residents' perceptions on what they valued most in clinical coaches, and to explore trainees' ideas about how to optimize this role.

DESIGN

We performed an exploratory qualitative study, using 45 minutes semi-structured interviews. We did a thematic analysis of the interview transcripts using both inductive and deductive coding.

PARTICIPANTS

We invited and interviewed 5 surgical and 5 nonsurgical residents, and 3 surgical and 3 nonsurgical attending staff. Residents were recruited from all post graduate levels and from a variety of programs.

SETTING

Our study was done in a large tertiary teaching hospital.

RESULTS

Residents perceived that they have a significant role as coaches for junior learners, different from the attending's role. The proximity between the coach and the coaches leads to a different supervisor-learner rapport. This was of benefit as learners described feeling more comfortable making mistakes and seeking feedback, which potentiates effective coaching. Residents reported feeling that it was easier to coach their recently-acquired skills as the subtleties of the tasks and the troubleshooting were fresher in memory. Residents expressed appreciating a coach who values autonomy and does not intervene except when patient safety is at risk. Strategies identified to further optimize residents' role as coaches include placing coaching as a priority, ensuring dedicated time, and offering teaching sessions on coaching.

CONCLUSIONS

Residents have distinct roles as coaches, driven by their recent experience being coached and as near peers. More research is needed to evaluate concrete measures to optimize residents' role as coaches and to improve their coaching skills.

The Evidence Basis for Learning Theory and Technology in Surgical Skills Training

Orthopaedic trainees face a complex and challenging training environment that is currently becoming more competency driven. Associated with these changes are an increasing introduction and use of a variety of technologically driven surgical training augments. Although these new learning resources can positively transform the educational environment, they must be used appropriately by both learners and educators. To aid in this, we review learning theories because they apply to surgical skills training and highlight recent surgical training evidence that demonstrates how technology use can be optimized to promote surgical learning, with an emphasis on procedural learning theory and cognitive load theory. Specifically, we review the evidence demonstrating the importance of targeting technology to a learner's experience level and methods to optimize cognitive load by managing intrinsic load, minimizing extraneous load, and maximizing germane load.

Education of physiotherapists improves inter-rater reliability in lumbar spine motor control tests: A randomized controlled trial

OBJECTIVES

To determine the effect of 1-h education session, compared with no education, on physiotherapists' (PTs) inter-rater reliability in two lumbar spine motor control tests (MCTs): waiter's bow (WB) and sitting knee extension (SKE). To determine whether reliability at baseline and the effect of education are affected by PT's clinical experience, knowledge and experience of MCTs, and post-graduate education in manual therapy.

DESIGN

Randomized controlled trial.

PARTICIPANTS

54 PTs.

INTERVENTION

The experimental group (EG) took part in a 1-h group education session. Control group (CG) had no intervention.

MEASUREMENTS

At baseline and after EG had completed the education session, therapists rated 40 SKE and 40 WB video recordings.

ANALYSIS

Changes in Fleiss' kappa were compared between groups. Differences >0.1 in kappa values were considered meaningful. Regression analysis was used to evaluate the effect of therapist's characteristics on inter-rater reliability at baseline and changes in inter-rater reliability.

RESULTS

Education had significant and meaningful effect on reliability compared with no education. WB kappa values improved from 0.36 to 0.63 in the experimental group and from .39 to .46 in the control group. SKE kappa values improved from .50 to .71 in the EG and from 0.49 to 0.57 in the CG. None of PTs' characteristics affected reliability at baseline or education effects.

CONCLUSION

The effect of 1-h group education session for physiotherapists on inter-rater reliability in MCTs is significant and meaningful. Provision of education for PTs in performing observational tests would improve inter-rater reliability, resulting in improved treatment planning and outcome evaluation.

Operative Coaching for General Surgery Residents: Review of Implementation Requirements

Operative coaching offers a unique opportunity to strengthen surgery residents' skill sets and practice readiness. However, institutional organizational capacity may influence the ability to successfully implement and sustain a coaching program. This review concentrates on the implementation requirements as they relate to institutional organizational capacity to help evaluate and determine if adopting such a coaching model is feasible. We searched English-language, peer-reviewed articles concerning operative coaching of general surgery residents between 2000 and 2020 with the MEDLINE database. The abstracts of 267 identified articles were further screened based on the presence of 2 inclusion criteria: general surgery residents and operative coaching. Then we summarized the reported implementation requirements. Findings revealed the implementation requirements (ie people, processes, technology/support resources, physical resources, and organizational systems) of 3 major types of resident operative coaching models were different. Video-assisted coaching faces the most barriers to implementation followed by video-based coaching; in-person coaching encounters the least barriers. Six questions are generated helping residency education leaders assess their readiness for an operative coaching program. Evaluation of the implementation requirements of a desired coaching program using the 5 organizational capacity elements is recommended to ensure the residency's ability to achieve a successful and sustainable program.

Is Physician Mentorship Associated With the Occurrence of Adverse Patient Safety Events?

BACKGROUND

Mentorship has been identified as a beneficial practice for doctors and key aspect of continuing professional development, associated with a number of potential clinical and nonclinical gains. The likely contribution of mentorship to enhancing patient safety is acknowledged, but there is a dearth of empirical studies that attempt to make associations between the impact of mentorship for physicians on patient safety outcomes. This article begins to fill this gap by exploring whether a physician with a mentor reports having fewer near-misses or adverse events, compared with a physician with no mentor.

METHODS

An online survey was administered to fellows and members of the Royal College of Physicians London using their membership database in April 2013. Adverse events and near misses are modeled as two separate binary variables using a logit regression framework with "having a mentor" being the main covariate. The marginal effect of this covariate captures the effect of mentorship on adverse events.

RESULTS

A total of 1755 doctors (37% female) responded who represented all internal medical specialties. Our results show that compared with physicians with no mentor, the probability of getting involved in an adverse event or near miss is reduced by 12.69% (95% confidence interval = -17.41 to -7.98) and 11.12% (95% confidence interval = -15.84 to -6.41) for physicians with a mentor.

CONCLUSIONS

Having a mentor may contribute toward minimizing preventable harm to patients, which is a priority for health systems internationally, but longer-term studies of mentorship are necessary to determine the aspects of mentorship that are particularly important for enhancing patient safety outcomes.

Reliability and Quality of YouTube Videos on Ultrasound-Guided Brachial Plexus Block: A Programmatical Review

BACKGROUND

Ultrasound-guided regional anesthesia has gained popularity over the last decade. This study aimed to assess whether YouTube videos sufficiently serve as an adjunctive tool for learning how to perform an ultrasound-guided brachial plexus block (BPB).

METHODS

All YouTube videos were classified, based on their sources, as either academic, manufacturer, educational, or individual videos. The metrics, accuracy, utility, reliability (using the Journal of American Medical Association Score benchmark criteria (JAMAS)), and educational quality (using the Global Quality Score (GQS) and Brachial Plexus Block Specific Quality Score (BSQS)) were validated.

RESULTS

Here, 175 videos were included. Academic (1.19 ± 0.62, mean ± standard deviation), manufacturer (1.17 ± 0.71), and educational videos (1.15 ± 0.76) had better JAMAS accuracy and reliability than individual videos (0.26 ± 0.67) (p < 0.001). Manufacturer (11.22 ± 1.63) and educational videos (10.33 ± 3.34) had a higher BSQS than individual videos (7.32 ± 4.20) (p < 0.001). All sources weakly addressed the equipment preparation and post-procedure questions after BSQS analysis.

CONCLUSIONS

The reliability and quality of ultrasound-guided BPB videos differ depending on their source. As YouTube is a useful educational platform for learners and teachers, global societies of regional anesthesiologists should set a standard for videos.

Maintaining technical proficiency in senior surgical fellows during the COVID-19 pandemic through virtual teaching

Objectives

The novel coronavirus (COVID-19) pandemic resulted in a severe reduction in operative opportunities for trainees. We hypothesized that augmenting independent practice with a bench model of vascular anastomoses using regular videoconferences and individual feedback would provide meaningful benefit in maintenance of technical skills in senior lung transplant surgical fellows.

Methods

A lung transplantation virtual technical skills course was developed. Surgical fellows were provided with a bench model and surgical instruments. Using a virtual communication platform, teaching sessions were held twice weekly, and fellows performed an anastomosis on camera. Video recordings were reviewed and critiqued by attending staff. At the end of the 3-month course, participants were surveyed about their experience. Warm ischaemic time was compared between fellows' five most recent cases before and after the pandemic.

Results

Seven senior surgical fellows participated and provided feedback. Fellows had graduated medical school an average of 14 years prior to fellowship, and spent an average of 5 hours (range 1.3 - 15 hours) of home practice. Five of seven (71%) participants reported improvement in their technical skills and increased confidence in performing a lung transplant. No significant difference in warm ischaemic time in procedures performed by fellows was identified (70.3 minutes pre-pandemic vs. 68.3 minutes post pandemic, p = 0.68).

Conclusions

A program of virtual technical skills teaching, individual video coaching, and independent practice provided benefit in maintaining technical skills in lung transplant surgical fellows during the COVID-19 pandemic, when equivalent operative experience was unavailable. Lessons learned from this exceptional time can be used to create simulation curricula for senior trainees.

Application of technology to educational needs in surgery

Innovations in surgical education follow advancing clinical technology. New surgical methods have prompted demand for systematic methods to leverage computing power and internet tools to achieve proficiency-based training goals. Virtual reality, high-fidelity patient simulation, web-based resources to facilitate performance assessment, and telementoring have become mainstream practices, although patient outcomes benefits are not well studied. Remote virtual meeting and mentoring have had transformative effects on resident experiences, the full effects of which remain to be seen.

Supporting laparoscopic general surgery training with digital technology: The United Kingdom and Ireland paradigm

Surgical training in the UK and Ireland has faced challenges following the implementation of the European Working Time Directive and postgraduate training reform. The health services are undergoing a digital transformation; digital technology is remodelling the delivery of surgical care and surgical training. This review aims to critically evaluate key issues in laparoscopic general surgical training and the digital technology such as virtual and augmented reality, telementoring and automated workflow analysis and surgical skills assessment. We include pre-clinical, proof of concept research and commercial systems that are being developed to provide solutions. Digital surgical technology is evolving through interdisciplinary collaboration to provide widespread access to high-quality laparoscopic general surgery training and assessment. In the future this could lead to integrated, context-aware systems that support surgical teams in providing safer surgical care.

Feasibility of Surgeon-Delivered Audit and Feedback Incorporating Peer Surgical Coaching to Reduce Fistula Incidence following Cleft Palate Repair: A Pilot Trial

BACKGROUND

Improving surgeons' technical performance may reduce their frequency of postoperative complications. The authors conducted a pilot trial to evaluate the feasibility of a surgeon-delivered audit and feedback intervention incorporating peer surgical coaching on technical performance among surgeons performing cleft palate repair, in advance of a future effectiveness trial.

METHODS

A nonrandomized, two-arm, unblinded pilot trial enrolled surgeons performing cleft palate repair. Participants completed a baseline audit of fistula incidence. Participants with a fistula incidence above the median were allocated to an intensive feedback intervention that included selecting a peer surgical coach, observing the coach perform palate repair, reviewing operative video of their own surgical technique with the coach, and proposing and implementing changes in their technique. All others were allocated to simple feedback (receiving audit results). Outcomes assessed were proportion of surgeons completing the baseline audit, disclosing their fistula incidence to peers, and completing the feedback intervention.

RESULTS

Seven surgeons enrolled in the trial. All seven completed the baseline audit and disclosed their fistula incidence to other participants. The median baseline fistula incidence was 0.4 percent (range, 0 to 10.5 percent). Two surgeons were unable to receive the feedback intervention. Of the five remaining surgeons, two were allocated to intensive feedback and three to simple feedback. All surgeons completed their assigned feedback intervention. Among surgeons receiving intensive feedback, fistula incidence was 5.9 percent at baseline and 0.0 percent following feedback (adjusted OR, 0.98; 95 percent CI, 0.44 to 2.17).

CONCLUSION

Surgeon-delivered audit and feedback incorporating peer coaching on technical performance was feasible for surgeons.

Validation of a cost-effective appendicectomy model for surgical training

BACKGROUND

Appendicitis is a commonly occurring condition worldwide. The gold standard treatment is appendicectomy. Although training models are commercially available for this procedure, they are often associated with high cost. Here we present a cost-effective model.

AIM

To establish construct validity of a cost-effective laparoscopic appendicectomy simulation model.

METHODS

Three groups of surgeons were recruited; novices (n = 31), of intermediate expertise (n = 13) and experts (n = 5) and asked to perform a simulated laparoscopic appendicectomy using the new model. Their performance was assessed by a faculty member and compared between the three groups using a validated scoring system (Global Operative Assessment of Laparoscopic Skills [GOALS] score).

RESULTS

One-way ANOVA test showed a significant difference in task performance between groups (p < 0.0001). Post-hoc comparisons after the application of Bonferroni correction (statistically significant p value <0.017) demonstrate a significant difference in performance between all groups for all GOALS categories as well as the total score. Effect size calculations showed that experience level had moderate (Eta-squared >0.5 and <0.8) and significant (>0.8) impact on the performance of the simulated procedure.

CONCLUSION

The model described in this study is cost-effective, valid and can adequately simulate appendicectomy. The authors recommend inclusion of this model to postgraduate surgical training.

Virtual coach: the next tool in functional endoscopic sinus surgery education

BACKGOUND

Functional endoscopic sinus surgery (FESS) can be challenging as novices become accustomed to handling endoscopes and instruments while navigating complex sinonasal anatomy. Experts demonstrate fluid and efficient motions when addressing pathology. The training process from novice to expert relies on hands-on experience in cadaveric laboratories and preceptorship models that require significant time and expense. This study aims to validate the use of a virtual coach to guide users step-by-step through a basic FESS.

METHODS

Seventeen surgeons were grouped into novice (n = 10) and expert (n = 7) based on self-reported levels of surgical experience. Users were trained using the maxillary antrostomy module in the Neurorhinological Surgery (NRS) simulator combining the physical craniofacial model with virtual reality (VR)-tracked surgical instruments in the VR operating room. The virtual coach guided the user using surgical videos, auditory, and visual cues. The coach recorded data for each subject including the number of times borders of the nasal cavity were encountered and time to completion. Users graded the usefulness of the virtual coach on a Likert questionnaire.

RESULTS

Face validity of our NRS simulator was replicated by user questionnaires, and construct validity replicated by differentiation between novice and expert level surgeons (p < 0.01). Novices contacted a significantly higher number of anatomic borders (n = 17) and had a longer operative time (t = 370 seconds). All users reported high scores on the benefit and usefulness of the virtual coach.

CONCLUSION

The virtual coach provides a useful tool to enhance FESS education by providing objective real-time data in a novel mixed-reality surgical environment.

Video-based coaching in surgical education: a systematic review and meta-analysis

BACKGROUND

In the era of competency-based surgical education, VBC has gained increased attention and may enhance the efficacy of surgical education. The objective of this systematic review was to summarize the existing evidence of video-based coaching (VBC) and compare VBC to traditional master-apprentice-based surgical education.

METHODS

We performed a systematic review and meta-analysis of randomized controlled trials (RCT) assessing VBC according to the PRISMA and Cochrane guidelines. The MEDLINE, EMBASE, and COCHRANE and Researchgate databases were searched for eligible manuscripts. Standard mean difference (SMD) of performance scoring scales was used to assess the effect of VBC versus traditional training without VBC (control).

RESULTS

Of 627 studies identified, 24 RCTs were eligible and evaluated. The studies included 778 surgical trainees (n = 386 VBC vs. n = 392 control). 13 performance scoring scales were used to assess technical competence; OSATS-GRS was the most common (n = 15). VBC was provided preoperative (n = 11), intraoperative (n = 1), postoperative (n = 10), and perioperative (n = 2). The majority of studies were unstructured, where identified coaching frameworks were PRACTICE (n = 1), GROW (n = 2) and Wisconsin Coaching Framework (n = 1). There was an effect on performance scoring scales in favor of VBC coaching (SMD 0.87, p < 0.001). In subgroup analyses, the residents had a larger relative effect (SMD 1.13; 0.61-1.65, p < 0.001) of VBC compared to medical students (SMD 0.43, 0.06-0.81, p < 0.001). The greatest source of potential bias was absence of blinding of the participants and personnel (n = 20).

CONCLUSION

Video-based coaching increases technical performance of medical students and surgical residents. There exist significant study and intervention heterogeneity that warrants further exploration, showing the need to structure and standardize video-based coaching tools.

Targeted surgical coaching can improve operative self-assessment ability: A single-blinded nonrandomized trial

BACKGROUND

The aim of the study was to investigate the effect of targeted surgical coaching on self-assessment of laparoscopic operative skill. Accurate self-assessment is vital for autonomous professional development. Surgical coaching can be used for performance improvement, but its role in this domain has been insufficiently investigated.

METHODS

This was a single site, nonrandomized, interrupted time series design trial. Participants were residents, fellows, and attending surgeons regularly performing laparoscopic general surgery operations. Each participant was enrolled in an individualized coaching program using review of personal and peer laparoscopic videos. The program involved 3 to 5 sessions over a period of 6 to 19 weeks. Coaching used case debriefing to target self-assessment proficiency, with a focus on objective interpretation of observations and facilitative capacity building. The primary outcome measure was self-assessment accuracy and correlation to expert ratings. The Objective Structured Assessment of Technical Skill global rating scale was utilized for evaluation.

RESULTS

Twelve participants were recruited and completed the coaching program. At baseline, there was no correlation between self-assessment and expert ratings. After completion of the coaching program there was correlation between self-assessment and expert ratings (P = .003) and improved self-assessment accuracy compared to baseline (P = .041).

CONCLUSION

This study has demonstrated that targeted coaching using video review of laparoscopic cases can improve operative self-assessment accuracy using the Objective Structured Assessment of Technical Skill.

Endoscopic surgery: talent or training?

PURPOSE

There are two groups of undergraduate students involved in endoscopic surgery with different degrees of experience: average and more experience. This study proves whether the subjective impression of the laparoscopic trainer is verifiable and which factors influence extreme talent.

METHODS

21 medical students of the eighth term of the University of Witten-Herdecke participated in the study. On their first course day, students got instructed in suturing and knot technique. They were then required to tie a maximum of five knots within 2 h. After a week, students repeated this procedure. Time used for tying knots was stopped.

RESULTS

Regarding the time students used for their first knots, great differences were provable (7-8 min, average 23 min). However, an adaption of the knotting time was noticed at the end of the first course day. This was confirmed during the second course day. Neither acquired factors (music, sport, etc.) nor individual factors (visual acuity, handedness, etc.) had any impact on the time used for knotting. Merely, one advantage was seen with the first knots with the factors of playing the guitar and having a more than 10-h surgical previous experience. Knotting times leveled off at 95% to less than 10 min, though.

DISCUSSION

Neither normally talented nor extremely talented junior surgeons could be noticed, and so could not the co-factors providing an advantage or disadvantage for surgery, respectively. All prospective surgeons can learn defined tasks (knots) by short interval training, and thus show similarly good results after a few repetitions.

A Video-Based Coaching Intervention to Improve Surgical Skill in Fourth-Year Medical Students

OBJECTIVE

For senior medical students pursuing careers in surgery, specific technical feedback is critical for developing foundational skills in preparation for residency. This pilot study seeks to assess the feasibility of a video-based coaching intervention to improve the suturing skills of fourth-year medical students.

DESIGN

Fourth-year medical students pursuing careers in surgery were randomized to intervention vs. control groups and completed 2 video recorded suture tasks. Students in the intervention group received a structured coaching session between consecutive suturing tasks, whereas students in the control group did not. Each coaching session consisted of a video review of the students' first suture task with a faculty member that provided directed feedback regarding technique. Following each suturing task, students were asked to self-assess their performance and provide feedback regarding the utility of the coaching session. All videos were deidentified and graded by independent faculty members for evaluation of suture technique.

SETTING

The University of Michigan Medical School in Ann Arbor, Michigan.

PARTICIPANTS

All fourth-year medical students pursuing careers in surgical specialties were contacted via e-mail for voluntary participation. In all, 16 students completed both baseline and follow up suture tasks.

RESULTS

All students who completed the coaching session would definitely recommend the session for other students. A total of 94% of the students strongly agreed that the exercise was a beneficial experience, and 75% strongly agreed that it improved their technical skills. Based on faculty grading, students in the intervention group demonstrated greater average improvements in bimanual dexterity compared to students in the control group; whereas students in the control group demonstrated greater average improvements in domains of efficiency and tissue handling compared to the intervention group. Based on student self-assessments, those in the intervention group had greater subjective improvements in all scored domains of bimanual dexterity, efficiency, tissue handling, and consistency compared to the control group. Subjective, free-response comments centered on themes of becoming more aware of hand movements when viewing their suturing from a new perspective, and the usefulness of the coaching advice.

CONCLUSIONS

This pilot study demonstrates the feasibility of a video-based coaching intervention for senior medical students. Students who participated in the coaching arm of the intervention noticed improvements in all domains of technical skill and noted that the experience was overwhelmingly positive. In summary, video-based review shows promise as an educational tool in medical education as a means to provide specific technical feedback.

Evaluating Surgical Coaching: A Mixed Methods Approach Reveals More Than Surveys Alone

OBJECTIVE

Traditionally, surgical educators have relied upon participant survey data for the evaluation of educational interventions. However, the ability of such subjective data to completely evaluate an intervention is limited. Our objective was to compare resident and attending surgeons' self-assessments of coaching sessions from surveys with independent observations from analysis of intraoperative and postoperative coaching transcripts.

DESIGN

Senior residents were video-recorded operating. Each was then coached by the operative attending in a 1:1 video review session. Teaching points made in the operating room (OR) and in post-OR coaching sessions were coded by independent observers using dialogue analysis then compared using t-tests. Participants were surveyed regarding the degree of teaching dedicated to specific topics and perceived changes in teaching level, resident comfort, educational assessments, and feedback provision between the OR and the post-OR coaching sessions.

SETTING

A single, large, urban, tertiary-care academic institution.

PARTICIPANTS

Ten PGY4 to 5 general surgery residents and 10 attending surgeons.

RESULTS

Although the reported experiences of teaching and coaching sessions by residents and faculty were similar (Pearson correlation coefficient = 0.88), these differed significantly from independent observations. Observers found that residents initiated a greater proportion of teaching points and had more educational needs assessments during coaching, compared to the OR. However, neither residents nor attendings reported a change between the 2 environments with regard to needs assessments nor comfort with asking questions or making suggestions. The only metric on which residents, attendings, and observers agreed was the provision of feedback.

CONCLUSIONS

Participants' perspectives, although considered highly reliable by traditional metrics, rarely aligned with analysis of the associated transcripts from independent observers. Independent observation showed a distinct benefit of coaching in terms of frequency and type of learning points. These findings highlight the importance of seeking different perspectives, data sources, and methodologies when evaluating clinical education interventions. Surgical education can benefit from increased use of dialogue analyses performed by independent observers, which may represent a viewpoint distinct from that obtained by survey methodology.

What do we know about coaching in medical education? A literature review

CONTEXT

Coaching has been employed successfully in the competitive sports, professional music, and business and corporate worlds. It is now emerging as a training modality in medical education.

OBJECTIVES

This paper reviews the current evidence on coaching strategies for doctors and medical students.

METHODS

An applied literature search was conducted in PubMed, MEDLINE and Web of Science. Predetermined definitions of coaching interventions and their evaluations were used to narrow 993 papers down to 21, which were included in the final review. The 21 papers were critiqued with reference to validated scoring metrics.

RESULTS

There are many papers discussing the merits of coaching in the world of medicine, but few evaluations of coaching interventions. Existing coaching methodologies can be broadly summarised into three categories: coaching for doctor/student well-being and resilience; coaching for improved non-technical skills, and coaching for technical skills. Identification of suitable papers for inclusion is complicated by theoretical uncertainty regarding coaching: many papers use the term as a synonym for teaching or mentoring. The strongest evidence for coaching lies in the teaching of technical skills.

CONCLUSIONS

There is weak- to medium-strength evidence to support coaching as a method of improving doctor well-being and enhancing non-technical skills, although the evidence base is limited as a whole. This review identifies strong evidence to support coaching as a method to improve technical skills. There is great scope for further studies investigating the power of coaching in medical students and doctors.

Improved laparoscopic skills in gynaecology trainees following a simulation-training program using take-home box trainers

BACKGROUND

Lack of time and access to equipment are recognised barriers to simulation training.

AIM

To investigate the effect of a take-home laparoscopic simulator training program on the laparoscopic skills of gynaecology trainees.

METHOD

Participants (n = 17 in 2015, n = 16 in 2016) were supplied with a box trainer, associated equipment and instructions on self-directed training. A program was designed and implemented in 2015 comprising of ten weekly laparoscopic skills tasks and modified in 2016 to eight monthly tasks. Half of the participants were randomly allocated a supervisor. Participants performed baseline and post-training assessments of laparoscopic skills in a box trainer task (thread transfer) and virtual reality simulator tasks (laparoscopic tubal ligation and bilateral oophorectomy).

RESULTS

Trainees in 2015 demonstrated an improvement in the median time to complete the laparoscopic tubal ligation task (baseline 124 s vs post-training 91 s, P = 0.041). There was no difference in the number of tubal ligation bleeding incidents, or in the time taken to complete the box trainer thread transfer task. In 2016 trainees demonstrated improvement in tubal ligation time (baseline 251 vs 71 post-training, P = 0.021) and bilateral oophorectomy time (baseline 891 s vs 504 post-training, P = 0.025). There was no significant difference in other outcome measures. There was no difference found in performance when groups were compared by supervisor allocation.

CONCLUSION

A take-home box trainer simulation-training program was associated with improvement in laparoscopic skills. This type of program may improve trainee access to simulation training.

Laparoscopic cholecystectomy: a prospective cohort study assessing the impact of grade of operating surgeon on operative time and 30-day morbidity

Introduction There is an increasing trend towards day case surgery for uncomplicated gallstone disease. The challenges of maximising training opportunities are well recognised by surgical trainees and the need to demonstrate timely progression of competencies is essential. Laparoscopic cholecystectomy provides the potential for excellent trainee learning opportunities. Our study builds upon previous work by assessing whether measures of outcome are still affected when cases are stratified based on procedural difficulty. Material and methods A prospective cohort study of all laparoscopic cholecystectomies conducted at a district general hospital between 2009 and 2014, performed under the care of a single consultant. The operative difficulty was determined using the Cuschieri classification. The primary endpoint was duration of operation. Secondary endpoints included length of hospital stay, delayed discharge rate and 30-day morbidity. Results A total of 266 laparoscopic cholecystectomies were performed during the study period. Mean operative time for all consultant-led cases was 52.5 minutes compared with 51.4 minutes for trainees (P = 0.67 unpaired t-test). When cases were stratified for difficulty, consultant-led cases were on average 5 minutes faster. Median duration of hospital stay was equivalent in both groups and there was no statistical difference in re-attendance (12.9% vs. 15.3% P = 0.59) or re-admission rates (3.2% vs. 8.1% P = 0.10) at 30 days. Conclusions Our study provides evidence that laparoscopic cholecystectomy provides a good training opportunity for surgical trainees without being detrimental to patient outcome. We recommend that, in selected patients, under consultant supervision, laparoscopic cholecystectomy can be performed primarily by the surgical trainee without impacting on patient outcome or theatre scheduling.

Using GoPro to Give Video-Assisted Operative Feedback for Surgery Residents: A Feasibility and Utility Assessment

OBJECTIVE

As an adjunct to simulation-based teaching, laparoscopic video-based surgical coaching has been an effective tool to augment surgical education. However, the wide use of video review in open surgery has been limited primarily due to technological and logistical challenges. The aims of our study were to (1) evaluate perceptions of general surgery (GS) residents on video-assisted operative instruction and (2) conduct a pilot study using a head-mounted GoPro in conjunction with the operative performance rating system to assess feasibility of providing video review to enhance operative feedback during open procedures.

DESIGN

GS residents were anonymously surveyed to evaluate their perceptions of oral and written operative feedback and use of video-based operative resources. We then conducted a pilot study of 10 GS residents to assess the utility and feasibility of using a GoPro to record resident performance of an arteriovenous fistula creation with an attending surgeon. Categorical variables were analyzed using the chi-square test.

SETTING

Academic, tertiary medical center.

PARTICIPANTS

GS residents and faculty.

RESULTS

A total of 59 GS residents were anonymously surveyed (response rate = 65.5%). A total of 40% (n = 24) of residents reported that structured evaluations rarely or never provided meaningful feedback. When feedback was received, 55% (n = 32) residents reported that it was only rarely or sometimes in regard to their operative skills. There was no significant difference in surveyed responses among junior postgraduate year (PGY 1-2), senior (PGY 3-4), or chief residents (PGY-5). A total of 80% (n = 8) of residents found the use of GoPro video review very or extremely useful for education; they also deemed video review more useful for operative feedback than written or communicative feedback. An overwhelming majority (90%, n = 9) felt that video review would lead to improved technical skills, wanted to review the video with the attending surgeon for further feedback, and desired expansion of this tool to include additional procedures.

CONCLUSIONS

Although there has been progress toward improving operative feedback, room for further improvement remains. The use of a head-mounted GoPro is a dynamic tool that provides high-quality video for operative review and has the potential to augment the training experience of GS residents. Future studies exploring a wide array of open procedures involving a greater number of trainees will be needed to further define the use of this resource.

Patient-specific mental rehearsal with interactive visual aids: a path worth exploring?

Background

Surgeons of today are faced with unprecedented challenges; necessitating a novel approach to pre-operative preparation which takes into account the specific tests each case poses. In this study, we examine patient-specific mental rehearsal for pre-surgical practice and assess whether this method has an additional effect when compared to generic mental rehearsal.

Methods

Sixteen medical students were trained how to perform a simulated laparoscopic cholecystectomy (SLC). After baseline assessments, they were randomised to two equal groups and asked to complete three SLCs involving different anatomical variants. Prior to each procedure, Group A practiced mental rehearsal with the use of a pre-prepared checklist and Group B mental rehearsal with the checklist combined with virtual models matching the anatomical variations of the SLCs. The performance of the two groups was compared using simulator provided metrics and competency assessment tool (CAT) scoring by two blinded assessors.

Results

The participants performed equally well when presented with a “straight-forward” anatomy [Group A vs. Group B—time sec: 445.5 vs. 496 p = 0.64—NOM: 437 vs. 413 p = 0.88—PL cm: 1317 vs. 1059 p = 0.32—per: 0.5 vs. 0 p = 0.22—NCB: 0 vs. 0 p = 0.71—DVS: 0 vs. 0 p = 0.2]; however, Group B performed significantly better [Group A vs. B Total CAT score—Short Cystic Duct (SCD): 20.5 vs. 26.31 p = 0.02 η² = 0.32—Double cystic Artery (DA): 24.75 vs. 30.5 p = 0.03 η² = 0.28] and committed less errors (Damage to Vital Structures—DVS, SCD: 4 vs. 0 p = 0.03 η²=0.34, DA: 0 vs. 1 p = 0.02 η² = 0.22). in the cases with more challenging anatomies.

Conclusion

These results suggest that patient-specific preparation with the combination of anatomical models and mental rehearsal may increase operative quality of complex procedures.

Individualized Performance Feedback to Surgical Residents Improves Appropriate Venous Thromboembolism Prophylaxis Prescription and Reduces Potentially Preventable VTE: A Prospective Cohort Study

OBJECTIVE

To investigate the effect of providing personal clinical effectiveness performance feedback to general surgery residents regarding prescription of appropriate venous thromboembolism (VTE) prophylaxis.

BACKGROUND

Residents are frequently charged with prescribing medications for patients, including VTE prophylaxis, but rarely receive individual performance feedback regarding these practice habits.

METHODS

This prospective cohort study at the Johns Hopkins Hospital compared outcomes across 3 study periods: (1) baseline, (2) scorecard alone, and (3) scorecard plus coaching. All general surgery residents (n = 49) and surgical patients (n = 2420) for whom residents wrote admission orders during the first 9 months of the 2013-2014 academic year were included. Outcomes included the proportions of patients prescribed appropriate VTE prophylaxis, patients with preventable VTE, and residents prescribing appropriate VTE prophylaxis for every patient, and results from the Accreditation Council for Graduate Medical Education resident survey.

RESULTS

At baseline, 89.4% of patients were prescribed appropriate VTE prophylaxis and only 45% of residents prescribed appropriate prophylaxis for every patient. During the scorecard period, appropriate VTE prophylaxis prescription significantly increased to 95.4% (P < 0.001). For the scorecard plus coaching period, significantly more residents prescribed appropriate prophylaxis for every patient (78% vs 45%, P = 0.0017). Preventable VTE was eliminated in both intervention periods (0% vs 0.35%, P = 0.046). After providing feedback, significantly more residents reported receiving data about practice habits on the Accreditation Council for Graduate Medical Education resident survey (87% vs 38%, P < 0.001).

CONCLUSIONS

Providing personal clinical effectiveness feedback including data and peer-to-peer coaching improves resident performance, and results in a significant reduction in harm for patients.

Consensus on structured training curriculum for transanal total mesorectal excision (TaTME)

BACKGROUND

The interest and adoption of transanal total mesorectal excision (TaTME) is growing amongst the colorectal surgical community, but there is no clear guidance on the optimal training framework to ensure safe practice for this novel operation. The aim of this study was to establish a consensus on a detailed structured training curriculum for TaTME.

METHODS

A consensus process to agree on the framework of the TaTME training curriculum was conducted, seeking views of 207 surgeons across 18 different countries, including 52 international experts in the field of TaTME. The process consisted of surveying potential learners of this technique, an international experts workshop and a final expert's consensus to draw an agreement on essential elements of the curriculum.

RESULTS

Appropriate case selection was strongly recommended, and TaTME should be offered to patients with mid and low rectal cancers, but not proximal rectal cancers. Pre-requisites to learn TaTME should include completion of training and accreditation in laparoscopic colorectal surgery, with prior experience in transanal surgery. Ideally, two surgeons should undergo training together in centres with high volume for rectal cancer surgery. Mentorship and multidisciplinary training were the two most important aspects of the curriculum, which should also include online modules and simulated training for purse-string suturing. Mentors should have performed at least 20 TaTME cases and be experienced in laparoscopic training. Reviewing the specimens' quality, clinical outcome data and entering data into a registry were recommended. Assessment should be an integral part of the curriculum using Global Assessment Scales, as formative assessment to promote learning and competency assessment tool as summative assessment.

CONCLUSIONS

A detailed framework for a structured TaTME training curriculum has been proposed. It encompasses various training modalities and assessment, as well as having the potential to provide quality control and future research initiatives for this novel technique.

Impact of laparoscopic surgery training laboratory on surgeon's performance

Minimally invasive surgery has been replacing the open standard technique in several procedures. Similar or even better postoperative outcomes have been described in laparoscopic or robot-assisted procedures when compared to open surgery. Moreover, minimally invasive surgery has been providing less postoperative pain, shorter hospitalization, and thus a faster return to daily activities. However, the learning curve required to obtain laparoscopic expertise has been a barrier in laparoscopic spreading. Laparoscopic surgery training laboratory has been developed to aid surgeons to overcome the challenging learning curve. It may include tutorials, inanimate model skills training (box models and virtual reality simulators), animal laboratory, and operating room observation. Several different laparoscopic courses are available with specific characteristics and goals. Herein, we aim to describe the activities performed in a dry and animal-model training laboratory and to evaluate the impact of different kinds of laparoscopic surgery training courses on surgeon’s performance. Several tasks are performed in dry and animal laboratory to reproduce a real surgery. A short period of training can improve laparoscopic surgical skills, although most of times it is not enough to confer laparoscopic expertise for participants. Nevertheless, this short period of training is able to increase the laparoscopic practice of surgeons in their communities. Full laparoscopic training in medical residence or fellowship programs is the best way of stimulating laparoscopic dissemination.

A study of crowdsourced segment-level surgical skill assessment using pairwise rankings

PURPOSE

Currently available methods for surgical skills assessment are either subjective or only provide global evaluations for the overall task. Such global evaluations do not inform trainees about where in the task they need to perform better. In this study, we investigated the reliability and validity of a framework to generate objective skill assessments for segments within a task, and compared assessments from our framework using crowdsourced segment ratings from surgically untrained individuals and expert surgeons against manually assigned global rating scores.

METHODS

Our framework includes (1) a binary classifier trained to generate preferences for pairs of task segments (i.e., given a pair of segments, specification of which one was performed better), (2) computing segment-level percentile scores based on the preferences, and (3) predicting task-level scores using the segment-level scores. We conducted a crowdsourcing user study to obtain manual preferences for segments within a suturing and knot-tying task from a crowd of surgically untrained individuals and a group of experts. We analyzed the inter-rater reliability of preferences obtained from the crowd and experts, and investigated the validity of task-level scores obtained using our framework. In addition, we compared accuracy of the crowd and expert preference classifiers, as well as the segment- and task-level scores obtained from the classifiers.

RESULTS

We observed moderate inter-rater reliability within the crowd (Fleiss' kappa, κ = 0.41) and experts (κ = 0.55). For both the crowd and experts, the accuracy of an automated classifier trained using all the task segments was above par as compared to the inter-rater agreement [crowd classifier 85 % (SE 2 %), expert classifier 89 % (SE 3 %)]. We predicted the overall global rating scores (GRS) for the task with a root-mean-squared error that was lower than one standard deviation of the ground-truth GRS. We observed a high correlation between segment-level scores (ρ ≥ 0.86) obtained using the crowd and expert preference classifiers. The task-level scores obtained using the crowd and expert preference classifier were also highly correlated with each other (ρ ≥ 0.84), and statistically equivalent within a margin of two points (for a score ranging from 6 to 30). Our analyses, however, did not demonstrate statistical significance in equivalence of accuracy between the crowd and expert classifiers within a 10 % margin.

CONCLUSIONS

Our framework implemented using crowdsourced pairwise comparisons leads to valid objective surgical skill assessment for segments within a task, and for the task overall. Crowdsourcing yields reliable pairwise comparisons of skill for segments within a task with high efficiency. Our framework may be deployed within surgical training programs for objective, automated, and standardized evaluation of technical skills.

Porcine cadaver organ or virtual-reality simulation training for laparoscopic cholecystectomy: a randomized, controlled trial

OBJECTIVES

As conventional laparoscopic procedural training requires live animals or cadaver organs, virtual simulation seems an attractive alternative. Therefore, we compared the transfer of training for the laparoscopic cholecystectomy from porcine cadaver organs vs virtual simulation to surgery in a live animal model in a prospective randomized trial.

DESIGN

After completing an intensive training in basic laparoscopic skills, 3 groups of 10 participants proceeded with no additional training (control group), 5 hours of cholecystectomy training on cadaver organs (= organ training) or proficiency-based cholecystectomy training on the LapMentor (= virtual-reality training). Participants were evaluated on time and quality during a laparoscopic cholecystectomy on a live anaesthetized pig at baseline, 1 week (= post) and 4 months (= retention) after training.

SETTING

All research was performed in the Center for Surgical Technologies, Leuven, Belgium.

PARTICIPANTS

In total, 30 volunteering medical students without prior experience in laparoscopy or minimally invasive surgery from the University of Leuven (Belgium).

RESULTS

The organ training group performed the procedure significantly faster than the virtual trainer and borderline significantly faster than control group at posttesting. Only 1 of 3 expert raters suggested significantly better quality of performance of the organ training group compared with both the other groups at posttesting (p < 0.01). There were no significant differences between groups at retention testing. The virtual trainer group did not outperform the control group at any time.

CONCLUSIONS

For trainees who are proficient in basic laparoscopic skills, the long-term advantage of additional procedural training, especially on a virtual but also on the conventional organ training model, remains to be proven.

Evolution of standardized clinical pathways: refining multidisciplinary care and process to improve outcomes of the surgical treatment of esophageal cancer

BACKGROUND

The aim of this study is to determine the effect of the implementation and evolution of a multidisciplinary esophagectomy care pathway on postoperative outcomes over a 20-year experience.

STUDY DESIGN

All patients undergoing esophagectomy for cancer between 1991 and 2012 were included. Patients were divided into four groups (Gp1 1991-1996, Gp2 1997-2002, Gp3 2003-2007, and Gp4 2008-2012).

RESULTS

Five hundred and ninety-five patients were included (Gp1 92, Gp2 159, Gp3 161, and Gp4 183). Age remained consistent over time; however, a progressive significant increase was observed in BMI and Charlson comorbidity index. Increases were also noted in patients with clinical stage III cancers, in the use of neoadjuvant chemoradiotherapy, in salvage esophagectomy and in the utilization of pretreatment jejunostomy. We observed a significant reduction in estimated blood loss (EBL) and operative room IV fluid administration (ORFA) during the study period. Median ICU stay and length of hospital stay (LOS) (10 (5-50) to 8 (5-115) days) decreased over time. In-hospital mortality (0.3 %) and postoperative complications remained consistent over time. cumulative sum (CUSUM) analysis showed that EBL, ORFA, and LOS all declined during the study period, reaching mean values at case 120, 310, and 175, respectively.

CONCLUSIONS

The results of this study show that process improvement within the pathway is likely more significant than the level of comorbidities, application of neoadjuvant chemoradiation, or technical approach in patients undergoing esophagectomy.