Assessment of psychomotor skills acquisition during laparoscopic cholecystectomy courses

Automated surgical skill assessment in colorectal surgery using a deep learning-based surgical phase recognition model

BACKGROUND

There is an increasing demand for automated surgical skill assessment to solve issues such as subjectivity and bias that accompany manual assessments. This study aimed to verify the feasibility of assessing surgical skills using a surgical phase recognition model.

METHODS

A deep learning-based model that recognizes five surgical phases of laparoscopic sigmoidectomy was constructed, and its ability to distinguish between three skill-level groups-the expert group, with a high Endoscopic Surgical Skill Qualification System (ESSQS) score (26 videos); the intermediate group, with a low ESSQS score (32 videos); and the novice group, with an experience of < 5 colorectal surgeries (27 videos)-was assessed. Furthermore, 1 272 videos were divided into three groups according to the ESSQS score: ESSQS-high, ESSQS-middle, and ESSQS-low groups, and whether they could be distinguished by the score calculated by multiple regression analysis of the parameters from the model was also evaluated.

RESULTS

The time for mobilization of the colon, time for dissection of the mesorectum plus transection of the rectum plus anastomosis, and phase transition counts were significantly shorter or less in the expert group than in the intermediate (p = 0.0094, 0.0028, and < 0.001, respectively) and novice groups (all p < 0.001). Mesorectal excision time was significantly shorter in the expert group than in the novice group (p = 0.0037). The group with higher ESSQS scores also had higher AI scores.

CONCLUSION

This model has the potential to be applied to automated skill assessments.

Laparoscopic cholecystectomy: a prospective cohort study assessing the impact of grade of operating surgeon on operative time and 30-day morbidity

Introduction There is an increasing trend towards day case surgery for uncomplicated gallstone disease. The challenges of maximising training opportunities are well recognised by surgical trainees and the need to demonstrate timely progression of competencies is essential. Laparoscopic cholecystectomy provides the potential for excellent trainee learning opportunities. Our study builds upon previous work by assessing whether measures of outcome are still affected when cases are stratified based on procedural difficulty. Material and methods A prospective cohort study of all laparoscopic cholecystectomies conducted at a district general hospital between 2009 and 2014, performed under the care of a single consultant. The operative difficulty was determined using the Cuschieri classification. The primary endpoint was duration of operation. Secondary endpoints included length of hospital stay, delayed discharge rate and 30-day morbidity. Results A total of 266 laparoscopic cholecystectomies were performed during the study period. Mean operative time for all consultant-led cases was 52.5 minutes compared with 51.4 minutes for trainees (P = 0.67 unpaired t-test). When cases were stratified for difficulty, consultant-led cases were on average 5 minutes faster. Median duration of hospital stay was equivalent in both groups and there was no statistical difference in re-attendance (12.9% vs. 15.3% P = 0.59) or re-admission rates (3.2% vs. 8.1% P = 0.10) at 30 days. Conclusions Our study provides evidence that laparoscopic cholecystectomy provides a good training opportunity for surgical trainees without being detrimental to patient outcome. We recommend that, in selected patients, under consultant supervision, laparoscopic cholecystectomy can be performed primarily by the surgical trainee without impacting on patient outcome or theatre scheduling.

Objective assessment of skills acquisition during laparoscopic surgery courses

BACKGROUND

The aim of this prospective study is to objectively assess the acquisition of skills of trainees attending laparoscopic surgery courses.

METHODS

Thirty-four junior surgical trainees had their laparoscopic skills assessed before and after attending 1 of 3 separate runs of 3-day core skills in laparoscopic surgery course. Nine control trainees were also included who did not attend the course. Three virtual tasks (camera navigation, hand-eye coordination, and 2-handed maneuver) were used from a virtual reality simulator (Simbionix) for assessment. Camera navigation was assessed by completion time and maintenance of horizontal view, whereas the other 2 tasks were assessed by completion time, path length (both hands), and the number of movements (both hands). A composite score of overall performance was calculated by combining all the 12 parameters.

RESULTS

The course significantly (P < 0.001) improved 91% of the junior trainees' precourse laparoscopic skills. Around 70% to 85% of the participants had improvement in skills in all the parameters following the course. The significant improvements were seen in 10 out of 12 task-specific parameters (P ≤ .004) except path length of the left hand. No significant improvement in skills was seen in any 1 of the 12 parameters for the control participants except for a slight reduction in performance matrics. Foundation and core trainees had acquired significantly (P = .02) more skills (23% improvement) than the specialist trainees (8% improvement). Overall acquired skills did not differ significantly in terms of age, sex, or dominant hand of trainees.

CONCLUSION

Objective validated methods can be used to demonstrate course efficacy in addition to providing participants with an insight into their skills. Junior trainees with little or no previous experience benefit the most from such courses irrespective of their age, sex, and dominant hand.

Preoperative warm-up using a virtual reality simulator

Even more experienced laparoscopic surgeons in this report became better prepared to perform a surgical procedure doing simple exercises on a virtual reality simulator before the actual procedure.

Background and Objectives:

All modern surgical procedures require a high level of cognitive and psychomotor skills achieved using different training methods, but could be influenced by fatigue and other psychological factors. We evaluated the effect of warm-up exercises on operative laparoscopic performances.

Methods:

The surgical team operated on a consecutive series of 20 patients with gallstones. Patients were randomly allocated in 2 groups: group A to be operated on without warm-up exercises and group B to be operated on after a short-term warm-up. All the patients were operated on by the same surgical team. The full-time records of the operation were analyzed by 2 independent reviewers. A modified simplified Global Rating Score (GRS) was used to assess the surgical procedures. A training module using the Lap Mentor simulator was designed for the warm-up.

Results:

Better performances were noted by both observers in group B only regarding “Respect for tissue” scores (3.75±0.16 vs 4.43±0.20, P=.021 and 3.87±0.22 vs 4.57±0.20, P=.041) achieving significant or marginally significant differences for all categories; GRS scores for “time and motion” and “overall impression” tend to be better after warm-up, but differences failed to reach statistical significance in our series.

Conclusion:

Surgeons, even the most experienced in laparoscopic surgery, can increase specific psychomotor skills associated with a laparoscopic environment by doing simple exercises on a virtual reality simulator, just before an operation. These improvements are reflected in more accurate handling of tissue during laparoscopic cholecystectomy.

Total laparoscopic hysterectomy: evaluation of an evidence-based educational strategy using a novel simulated suture and knot-tying challenge, the "holiotomy"

Objective. The purpose of this study was to evaluate perceptions of skills and practice patterns of gynecologists attending a course on total laparoscopic hysterectomy (TLH). This course employed extensive use of pelvic trainer boxes to accomplish the Holiotomy Challenge. The "Holiotomy Challenge" entailed suturing two plastic pieces with six figure-of-N sutures tied with four square knots each. Methods. A survey was administered before the course and 3 months later. Data were analyzed by paired t-tests, McNemar's Chi Squares, and ANCOVAs with significance set P < .05. Results. At baseline, 216 surgeons and at 3 months 102 surgeons returned the survey. Surgeons' self-perceptions of their skills significantly increased from 6.24 to 7.28. Their reports of their surgical practice at home revealed significantly increased rates of minimally invasive procedures, from 42% to 54%. Significantly more surgeons reported having the ability to close the vagina, or a small cystotomy or enterotomy. Participation in the cadaver lab and presence of their practice partner did not impact these rates. Conclusions. A comprehensive course employing laparoscopic surgical simulation focused on basic surgical skills essential to TLH has a positive impact on attendees' self-rated skill level and rate of laparoscopic approaches. Many had begun performing TLH after the course.

Tracking manikin tracheal intubation using motion analysis

OBJECTIVES

This study investigates whether laryngoscope motion-tracking technology can be used to differentiate expert versus novice providers' techniques during endotracheal intubation in infant manikins; this may help improve intubation techniques.

METHODS

Each of 11 experts and 11 novices intubated an infant manikin head (Laerdal Corp, Wappinger Falls, NY) 10 times. Laryngoscope motion was tracked using electromagnetic technology during: (1) time from acquisition of laryngoscope to oral insertion, (2) insertion to stabilization of laryngoscope, and (3) stabilization of laryngoscope to insertion of endotracheal tube and withdrawal of laryngoscope. There were 213/220 analyzable data files. Expert versus novice rate of success, laryngoscope blade-tip motion path length, handle angle at intubation, time in each phase, and motion of handle relative to manikin were compared.

RESULTS

Intubation success rate was greater for experts (105/105 = 100% vs novices 101/108 = 93.5%, P < 0.001). Expert path of motion in phase 2 was longer (mean, 39 vs 29 cm, P < 0.001). The mean difference in the laryngoscope handle angle relative to the manikin occiput was statistically significant (mean angle, -54.42 vs -56.63 degrees; P = 0.001) but within the equipment testing margin of error (2 degrees). Time from insertion to withdrawal of laryngoscope (phases 2 and 3 combined) was greater for experts (16.45 vs 13.15 seconds; P = 0.02). Both experts and novices "rocked" the laryngoscope to achieve laryngeal visualization.

CONCLUSIONS

It is feasible to track laryngoscope motion during manikin intubation comparing expert versus novice technique. Experts had a greater success rate, but longer path length, and took longer to achieve manikin intubation. Motion-tracking technology may provide an analytic tool to improve techniques of intubation.

Validation of orthopaedic bench models for trauma surgery

The aim of this study was to validate the use of three models of fracture fixation in the assessment of technical skills. We recruited 21 subjects (six experts, seven intermediates, and eight novices) to perform three procedures: application of a dynamic compression plate on a cadaver porcine model, insertion of an unreamed tibial intramedullary nail, and application of a forearm external fixator, both on synthetic bone models. The primary outcome measures were the Objective Structural Assessment of technical skills global rating scale on video recordings of the procedures which were scored by two independent expert observers, and the hand movements of the surgeons which were analysed using the Imperial College Surgical Assessment Device. The video scores were significantly different for the three groups in all three procedures (p < 0.05), with excellent inter-rater reliability (alpha = 0.88). The novice and intermediate groups specifically were significantly different in their performance with dynamic compression plate and intramedullary nails (p < 0.05). Movement analysis distinguished between the three groups in the dynamic compression plate model, but a ceiling effect was demonstrated in the intramedullary nail and external fixator procedures, where intermediates and experts performed to comparable standards (p > 0.6). A total of 85% (18 of 21) of the subjects found the dynamic compression model and 57% (12 of 21) found all the models acceptable tools of assessment. This study has validated a low-cost, high-fidelity porcine dynamic compression plate model using video rating scores for skills assessment and movement analysis. It has also demonstrated that Synbone models for the application of and intramedullary nail and an external fixator are less sensitive and should be improved for further assessment of surgical skills in trauma. The availability of valid objective tools of assessment of surgical skills allows further studies into improving methods of training.

Surgical simulator design and development

With the introduction of minimally invasive surgery (MIS), it became necessary to develop training methods to learn skills outside the operating room. Several training simulators have become commercially available, but fundamental research into the requirements for effective and efficient training in MIS is still lacking. Three aspects of developing a training program are investigated here: what should be trained, how it should be trained, and how to assess the results of training. In addition, studies are presented that have investigated the role of force feedback in surgical simulators. Training should be adapted to the level of behavior: skill-based, rule-based, or knowledge-based. These levels can be used to design and structure a training program. Extra motivation for training can be created by assessment. During MIS, force feedback is reduced owing to friction in the laparoscopic instruments and within the trocar. The friction characteristics vary largely among instruments and trocars. When force feedback is incorporated into training, it should include the large variation in force feedback properties as well. Training different levels of behavior requires different training methods. Although force feedback is reduced during MIS, it is needed for tissue manipulation, and therefore force application should be trained as well.

Retracting and seeking movements during laparoscopic goal-oriented movements. Is the shortest path length optimal?

Aims

Minimally invasive surgery (MIS) requires a high degree of eye–hand coordination from the surgeon. To facilitate the learning process, objective assessment systems based on analysis of the instruments’ motion are being developed. To investigate the influence of performance on motion characteristics, we examined goal-oriented movements in a box trainer. In general, goal-oriented movements consist of a retracting and a seeking phase, and are, however, not performed via the shortest path length. Therefore, we hypothesized that the shortest path is not an optimal concept in MIS.

Methods

Participants were divided into three groups (experts, residents, and novices). Each participant performed a number of one-hand positioning tasks in a box trainer. Movements of the instrument were recorded with the TrEndo tracking system. The movement from point A to B was divided into two phases: A-M (retracting) and M-B (seeking). Normalized path lengths (given in %) of the two phases were compared.

Results

Thirty eight participants contributed. For the retracting phase, we found no significant difference between experts [median (range) %: 152 (129–178)], residents [164 (126–250)], and novices [168 (136–268)]. In the seeking phase, we find a significant difference (<0.001) between experts [180 (172–247)], residents [201 (163–287)], and novices [290 (244–469)]. Moreover, within each group, a significant difference between retracting and seeking phases was observed.

Conclusions

Goal-oriented movements in MIS can be split into two phases: retracting and seeking. Novices are less effective than experts and residents in the seeking phase. Therefore, the seeking phase is characteristic of performance differences. Furthermore, the retracting phase is essential, because it improves safety by avoiding intermediate tissue contact. Therefore, the shortest path length, as presently used during the assessment of basic MIS skills, may be not a proper concept for analyzing optimal movements and, therefore, needs to be revised.

High fidelity simulation can discriminate between novice and experienced residents when assessing competency in patient care

BACKGROUND

High fidelity (HF) simulators have become more common in residency training programs.

AIMS

We hypothesized that high fidelity (HF) simulation-based assessment of patient care competency could differentiate novice from experienced residents.

METHODS

Prospective study of 44 emergency medicine residents. A simulated case of anaphylactic shock was administered to each participating resident utilizing a HF patient simulator. Management of the case required epinephrine, airway management including a surgical airway, and i.v. fluids. Data was recorded using a standardized form and stop watch.

PRIMARY OUTCOME

Time to completion of surgical airway.

SECONDARY OUTCOMES

Times to administration of epinephrine/attempt intubation/start surgical airway/complete case; checklist items: epinephrine as first action, pre-oxygenation, cricoid pressure and type of surgical airway.

RESULTS

Novice residents took significantly longer than the experienced residents to achieve our primary outcome, time to completion of surgical airway (621/512 sec; p = 0.03). The novice residents took significantly longer to achieve three of our secondary outcomes: time to start of surgical airway (534 versus 442 sec; p = 0.04), time to case completion (650 versus 513 sec; p = 0.006), and epinephrine as a first action (73% versus 100%; p = 0.02).

CONCLUSIONS

HF simulation-based assessment using objective measures, particularly time to action, discerned our novice from our experienced residents.