Face, content, and construct validity of the EndoViS training system for objective assessment of psychomotor skills of laparoscopic surgeons

Evaluation of skill acquisition characteristics depending on the size of a dry box

BACKGROUND

Few studies have analyzed the effect of the size of the working space in training on the acquisition of endoscopic skills. In this study, adult- and infant-sized dry boxes (DBs) were used to verify how the size of the working space in training affects forceps manipulation and learning curve.

MATERIAL AND METHODS

Seventy-two medical students were enrolled. The task was peg transfer. The training environment was divided into adult- and infant-sized DBs. Skill evaluations were also divided into adult- and infant-sized DBs (four groups in total). The forceps manipulation characteristics and task completion time were compared before and after training.

RESULTS

Regarding skill evaluations using adult-sized DBs, there were no significant differences between the infant- and adult-sized DB-trained groups. Regarding skill evaluations using infant-sized DBs, there were no significant differences between the groups before training. After training, there was no significant difference in the total path length or average acceleration of the forceps between the groups. However, the infant-sized DB-trained group had a significantly faster average forceps velocity and faster task completion time than the adult-sized DB-trained group.

CONCLUSION

Training with a small DB is more efficient in acquiring smoother and faster forceps manipulation in a small working space.

Haptic based fundamentals of laparoscopic surgery simulation for training with objective assessments

Force is crucial for learning psychomotor skills in laparoscopic tissue manipulation. Fundamental laparoscopic surgery (FLS), on the other hand, only measures time and position accuracy. FLS is a commonly used training program for basic laparoscopic training through part tasks. The FLS is employed in most of the laparoscopic training systems, including box trainers and virtual reality (VR) simulators. However, many laparoscopic VR simulators lack force feedback and measure tissue damage solely through visual feedback based on virtual collisions. Few VR simulators that provide force feedback have subjective force metrics. To provide an objective force assessment for haptic skills training in the VR simulators, we extend the FLS part tasks to haptic-based FLS (HFLS), focusing on controlled force exertion. We interface the simulated HFLS part tasks with a customized bi-manual haptic simulator that offers five degrees of freedom (DOF) for force feedback. The proposed tasks are evaluated through face and content validity among laparoscopic surgeons of varying experience levels. The results show that trainees perform better in HFLS tasks. The average Likert score observed for face and content validity is greater than 4.6 ± 0.3 and 4 ± 0.5 for all the part tasks, which indicates the acceptance of the simulator among subjects for its appearance and functionality. Face and content validations show the need to improve haptic realism, which is also observed in existing simulators. To enhance the accuracy of force rendering, we incorporated a laparoscopic tool force model into the simulation. We study the effectiveness of the model through a psychophysical study that measures just noticeable difference (JND) for the laparoscopic gripping task. The study reveals an insignificant decrease in gripping-force JND. A simple linear model could be sufficient for gripper force feedback, and a non-linear LapTool force model does not affect the force perception for the force range of 0.5-2.5 N. Further study is required to understand the usability of the force model in laparoscopic training at a higher force range. Additionally, the construct validity of HFLS will confirm the applicability of the developed simulator to train surgeons with different levels of experience.

Surgical tooltip motion metrics assessment using virtual marker: an objective approach to skill assessment for minimally invasive surgery

PURPOSE

Surgical skill assessment has primarily been performed using checklists or rating scales, which are prone to bias and subjectivity. To tackle this shortcoming, assessment of surgical tool motion can be implemented to objectively classify skill levels. Due to the challenges involved in motion tracking of surgical tooltips in minimally invasive surgeries, formerly used assessment approaches may not be feasible for real-world skill assessment. We proposed an assessment approach based on the virtual marker on surgical tooltips to derive the tooltip's 3D position and introduced a novel metric for surgical skill assessment.

METHODS

We obtained the 3D tooltip position based on markers placed on the tool handle. Then, we derived tooltip motion metrics to identify the metrics differentiating the skill levels for objective surgical skill assessment. We proposed a new tooltip motion metric, i.e., motion inconsistency, that can assess the skill level, and also can evaluate the stage of skill learning. In this study, peg transfer, dual transfer, and rubber band translocation tasks were included, and nine novices, five surgical residents and five attending general surgeons participated.

RESULTS

Our analyses showed that tooltip path length (p [Formula: see text] 0.007) and path length along the instrument axis (p [Formula: see text] 0.014) differed across the three skill levels in all the tasks and decreased by skill level. Tooltip motion inconsistency showed significant differences among the three skill levels in the dual transfer (p [Formula: see text] 0.025) and the rubber band translocation tasks (p [Formula: see text] 0.021). Lastly, bimanual dexterity differed across the three skill levels in all the tasks (p [Formula: see text] 0.012) and increased by skill level.

CONCLUSION

Depth perception ability (indicated by shorter tooltip path lengths along the instrument axis), bimanual dexterity, tooltip motion consistency, and economical tooltip movements (shorter tooltip path lengths) are related to surgical skill. Our findings can contribute to objective surgical skill assessment, reducing subjectivity, bias, and associated costs.

A systematic review on classification and assessment of surgical skill levels for simulation-based training programs

BACKGROUND

Nowadays, advances in medical informatics have made minimally invasive surgery (MIS) procedures the preferred choice. However, there are several problems with the education programs in terms of surgical skill acquisition. For instance, defining and objectively measuring surgical skill levels is a challenging process. Accordingly, the aim of this study is to conduct a literature review for an investigation of the current approaches for classifying the surgical skill levels and for identifying the skill training tools and measurement methods.

MATERIALS AND METHODS

In this research, a search is conducted and a corpus is created. Exclusion and inclusion criteria are applied by limiting the number of articles based on surgical education, training approximations, hand movements, and endoscopic or laparoscopic operations. To satisfy these criteria, 57 articles are included in the corpus of this study.

RESULTS

Currently used surgical skill assessment approaches have been summarized. Results show that various classification approaches for the surgical skill level definitions are being used. Besides, many studies are conducted by omitting particularly important skill levels in between. Additionally, some inconsistencies are also identified across the skill level classification studies.

CONCLUSION

In order to improve the benefits of simulation-based training programs, a standardized interdisciplinary approach should be developed. For this reason, specific to each surgical procedure, the required skills should be identified. Additionally, appropriate measures for assessing these skills, which can be defined in simulation-based MIS training environments, should be refined. Finally, the skill levels gained during the developmental stages of these skills, with their threshold values referencing the identified measures, should be redefined in a standardized manner.

Face, Content, and Construct Validity Evaluation of Simulation Models in General Surgery Laparoscopic Training and Education: A Systematic Review

BACKGROUND

Laparoscopic technical surgical skills (LTS) are considered a fundamental competence for General Surgery residents. Several simulation tools (ST) have been explored to develop LTS. Although a plethora of systematic reviews evaluate the translation of LTS developed in simulation to real surgery, there is a lack of evidence that clarifies effectiveness of different validated ST in acquisition of LTS in surgical residents. The aim of this systematic review (SR) is to summarize published evidence on ST validation used for surgery education and training.

METHODS

A protocol was published in PROSPERO. A SR was carried out following PRISMA guidelines. Complete published articles in English or Spanish that validate either content or construct, plus another form of validation of ST to acquire LTS in general surgery were included. Articles that used only one validation or did not validate an ST were excluded.

RESULTS

1052 publications were initially identified across all searched databases. Title review identified 204 studies eligible for full text screening. 10 studies were included for final review. Two studies assessed both face and content, 4 face and construct, and 4 face, content and construct validity. None of the studies presented comparable outcomes due to metrics variation and scores used for the validation strategies.

CONCLUSIONS

This study assessed validated laparoscopic simulation models, particularly in content and construct validity. Articles reported an increased use of simulation models in laparoscopic training with positive feedback from trainees, but few studies reported validation of training model. Validation strategies are not standardized, limiting comparability between them.

Objective psychomotor laparoscopic skills evaluation using a low-cost wearable device based on accelerometry: construct and concurrent validity study

BACKGROUND

Motion analysis of surgical maneuvers provides useful quantitative information for the objective evaluation of the surgeons. However, surgical simulation laboratories for laparoscopic training do not usually integrate devices that help quantify the level of skills of the surgeons due to their limited resources and the high costs of new technologies. The purpose of this study is to present the construct and concurrent validity of a low-cost motion tracking system, based on a wireless triaxial accelerometer, employed to objectively evaluate psychomotor skills of surgeons during laparoscopic training.

METHODS

An accelerometry system, a wireless three-axis accelerometer with appearance of wristwatch, was placed on the dominant hand of the surgeons to register the motion during the laparoscopy practice with the EndoViS simulator, which simultaneously recorded the motion of the laparoscopic needle driver. This study included the participation of 30 surgeons (6 experts, 14 intermediates and 10 novices) who performed the task of intracorporeal knot-tying suture. Using 11 motion analysis parameters (MAPs), the performance of each participant was assessed. Subsequently, the scores of the three groups of surgeons were statistically analyzed. In addition, a validity study was conducted comparing the metrics between the accelerometry-tracking system and the EndoViS hybrid simulator.

RESULTS

Construct validity was achieved for 8 of the 11 metrics examined with the accelerometry system. Concurrent validity demonstrated that there is a strong correlation between the results of the accelerometry system and the EndoViS simulator in 9 of 11 parameters, showing reliability of the accelerometry system as an objective evaluation method.

CONCLUSION

The accelerometry system was successfully validated. This method is potentially useful to complement the objective evaluation of surgeons during laparoscopic practice in training environments such as box-trainers and simulators.

Assessment of Minimally Invasive Suturing Skills: Is Instrument Tracking an Accurate Prediction?

Introduction: Minimally invasive surgery (MIS) suturing demands advanced surgical skills. Therefore, it is important these skills are adequately trained and assessed. Assessment and feedback can consist of judgments and scores of expert observers or objective parameters using instrument tracking. The aim of this study was to determine to what extent objective parameters correspond to expert assessment. Methods: Participants performed an intracorporeal suturing task on the EoSim simulator repeatedly (maximum 20 repetitions) during training. The best discriminating parameters, which previously shown construct validation, were combined into a composite score, using regression analysis. All videos were blinded and assessed by 2 independent reviewers using the validated laparoscopic suturing competency assessment tool (LS-CAT). These scores were compared with the composite score. Results: A 100 videos of 16 trainees, during separate points on their learning curve, and 8 experts were used. The parameters "time" and "distance" were statistically significantly correlated with all LS-CAT domains. The composite score (calculated from "time" and "distance") showed improvement between the first and the last knot (57% versus 94%, P < .001). Also the LS-CAT score improved (28 versus 17, P < .001). However, the correlation of the composite score with the LS-CAT score was weak (R: 0.351), with an accuracy of 55/100 when pooling the outcomes based on inadequate, adequate, or good performance. Conclusion: Instrument tracking parameters (using Surgtrac) could give an indication of the skill level, however, it missed important elements, essential for reliable assessment. Therefore, expert assessment remains superior to determine the skill level in MIS suturing skills.

Needle path planning in semiautonomous and teleoperated robot-assisted epidural anaesthesia procedure: A proof of concept

BACKGROUND

Epidural anaesthesia is a Percutaneous Procedure (PP) which plays a crucial role in surgical procedures, where accurate needle insertion is still challenging. The objective of this work is to present a Tuohy needle path planning, which allows an anaesthesiologist to drive semiautonomously, with the assistance of a teleoperated robot, the tip of the needle during this PP.

METHODS

We capture, analysed and modelled the anaesthetist hands' motion during the execution of this procedure, by synthetising, programing and simulating a parametrised and normalised kinematic constrains dependent on an insertion variable in a virtual robot.

RESULTS

Two preoperative path planning models were obtained, which provide a teleoperated robot with kinematic constraints to semiautonomously drive a Tuohy needle in the epidural anaesthesia procedure.

CONCLUSIONS

A semiautonomous robot can assist in the execution of this PP using the kinematic constraints obtained from the study of the movement of a specialist's hands.

Integration of Comprehensive Metrics into the PsT1 Neuroendoscopic Training System

OBJECTIVE

Metric-based surgical training can be used to quantify the level and progression of neurosurgical performance to optimize and monitor training progress. Here we applied innovative metrics to a physical neurosurgery trainer to explore whether these metrics differentiate between different levels of experience across different tasks.

METHODS

Twenty-four participants (9 experts, 15 novices) performed 4 tasks (dissection, spatial adaptation, depth adaptation, and the A-B-A task) using the PsT1 training system. Four performance metrics (collision, precision, dissected area, and time) and 6 kinematic metrics (dispersion, path length, depth perception, velocity, acceleration, and motion smoothness) were collected.

RESULTS

For all tasks, the execution time (t) of the experts was significantly lower than that of novices (P < 0.05). The experts performed significantly better in all but 2 of the other metrics, dispersion and sectional area, corresponding to the A-B-A task and dissection task, respectively, for which they showed a nonsignificant trend towards better performance (P = 0.052 and P = 0.076, respectively).

CONCLUSIONS

It is possible to differentiate between the skill levels of novices and experts according to parameters derived from the PsT1 platform, paving the way for the quantitative assessment of training progress using this system. During the current coronavirus disease 2019 pandemic, neurosurgical simulators that gather surgical performance metrics offer a solution to the educational needs of residents.

Motion analysis for better understanding of psychomotor skills in laparoscopy: objective assessment-based simulation training using animal organs

Background

Our aim was to characterize the motions of multiple laparoscopic surgical instruments among participants with different levels of surgical experience in a series of wet-lab training drills, in which participants need to perform a range of surgical procedures including grasping tissue, tissue traction and dissection, applying a Hem-o-lok clip, and suturing/knotting, and digitize the level of surgical competency.

Methods

Participants performed tissue dissection around the aorta, dividing encountered vessels after applying a Hem-o-lok (Task 1), and renal parenchymal closure (Task 2: suturing, Task 3: suturing and knot-tying), using swine cadaveric organs placed in a box trainer under a motion capture (Mocap) system. Motion-related metrics were compared according to participants’ level of surgical experience (experts: 50 ≤ laparoscopic surgeries, intermediates: 10–49, novices: 0–9), using the Kruskal–Wallis test, and significant metrics were subjected to principal component analysis (PCA).

Results

A total of 15 experts, 12 intermediates, and 18 novices participated in the training. In Task 1, a shorter path length and faster velocity/acceleration/jerk were observed using both scissors and a Hem-o-lok applier in the experts, and Hem-o-lok-related metrics markedly contributed to the 1st principal component on PCA analysis, followed by scissors-related metrics. Higher-level skills including a shorter path length and faster velocity were observed in both hands of the experts also in tasks 2 and 3. Sub-analysis showed that, in experts with 100 ≤  cases, scissors moved more frequently in the “close zone (0  ≤ to < 2.0 cm from aorta)” than those with 50–99 cases.

Conclusion

Our novel Mocap system recognized significant differences in several metrics in multiple instruments according to the level of surgical experience. “Applying a Hem-o-lok clip on a pedicle” strongly reflected the level of surgical experience, and zone-metrics may be a promising tool to assess surgical expertise. Our next challenge is to give completely objective feedback to trainees on-site in the wet-lab.

Electronic supplementary material

The online version of this article (10.1007/s00464-020-07940-7) contains supplementary material, which is available to authorized users.

Construct validity of the SurgForce system for objective assessment of laparoscopic suturing skills

BACKGROUND

Laparoscopic surgery requires a new set of skill to be learned by the surgeons, of which the most relevant is tissue manipulation. Excessive forces applied to the tissue can cause rupture during manipulation or ischemia when confronting both sides of the tissue. The aim of this study is to establish the construct validity of the SurgForce system for objective assessment of advanced laparoscopic skills, based on the force signal generated during suture tasks, and the development of force parameters for evaluating tissue handling interaction.

METHODS

The SurgForce system, a tissue handling training device that measures dynamic force, was used to capture the force generated by surgeons with different levels of laparoscopic experience. For construct validity, 37 participants were enrolled in this study: 19 medical students, 12 residents of surgical specialties and 6 expert surgeons. All participants performed an intracorporeal knotting suture task over a synthetic tissue pad with a laparoscopic box-trainer. The force performance of the participants was analyzed using 11 force-based parameters with the application of the SurgForce system. Statistical analysis was performed between novice, intermediate, and expert groups using a Kruskal-Wallis test, and between the pairs of groups using a Mann-Whitney U-test.

RESULTS

Overall, 9 of the 11 force-related parameters showed significant differences between the three study groups. Results between the pairs of groups presented significant differences in 5 force parameters proposed. Construct validity results demonstrated that the SurgForce system was able to differentiate force performance between surgeons with different levels of laparoscopic experience.

CONCLUSION

The SurgForce system was successfully validated. This force system showed its potential to measure the force exerted on tissue for objective assessment of tissue handling skills in suturing tasks. Furthermore, its compact design allows the use of this device in conventional laparoscopic box-trainers.

Use of a Low-Cost Portable 3D Virtual Reality Gesture-Mediated Simulator for Training and Learning Basic Psychomotor Skills in Minimally Invasive Surgery: Development and Content Validity Study

Background

Simulation in virtual environments has become a new paradigm for surgeon training in minimally invasive surgery (MIS). However, this technology is expensive and difficult to access.

Objective

This study aims first to describe the development of a new gesture-based simulator for learning skills in MIS and, second, to establish its fidelity to the criterion and sources of content-related validity evidence.

Methods

For the development of the gesture-mediated simulator for MIS using virtual reality (SIMISGEST-VR), a design-based research (DBR) paradigm was adopted. For the second objective, 30 participants completed a questionnaire, with responses scored on a 5-point Likert scale. A literature review on the validity of the MIS training-VR (MIST-VR) was conducted. The study of fidelity to the criterion was rated using a 10-item questionnaire, while the sources of content-related validity evidence were assessed using 10 questions about the simulator training capacity and 6 questions about MIS tasks, and an iterative process of instrument pilot testing was performed.

Results

A good enough prototype of a gesture-based simulator was developed with metrics and feedback for learning psychomotor skills in MIS. As per the survey conducted to assess the fidelity to the criterion, all 30 participants felt that most aspects of the simulator were adequately realistic and that it could be used as a tool for teaching basic psychomotor skills in laparoscopic surgery (Likert score: 4.07-4.73). The sources of content-related validity evidence showed that this study’s simulator is a reliable training tool and that the exercises enable learning of the basic psychomotor skills required in MIS (Likert score: 4.28-4.67).

Conclusions

The development of gesture-based 3D virtual environments for training and learning basic psychomotor skills in MIS opens up a new approach to low-cost, portable simulation that allows ubiquitous learning and preoperative warm-up. Fidelity to the criterion was duly evaluated, which allowed a good enough prototype to be achieved. Content-related validity evidence for SIMISGEST-VR was also obtained.

Development and Validation of a Homemade, Low-Cost Laparoscopic Simulator for Resident Surgeons (LABOT)

Several studies have demonstrated that training with a laparoscopic simulator improves laparoscopic technical skills. We describe how to build a homemade, low-cost laparoscopic training simulator (LABOT) and its validation as a training instrument. First, sixty surgeons filled out a survey characterized by 12 closed-answer questions about realism, ergonomics, and usefulness for surgical training (global scores ranged from 1—very insufficient to 5—very good). The results of the questionnaires showed a mean (±SD) rating score of 4.18 ± 0.65 for all users. Then, 15 students (group S) and 15 residents (group R) completed 3 different tasks (T1, T2, T3), which were repeated twice to evaluate the execution time and the number of users’ procedural errors. For T1, the R group had a lower mean execution time and a lower rate of procedural errors than the S group; for T2, the R and S groups had a similar mean execution time, but the R group had a lower rate of errors; and for T3, the R and S groups had a similar mean execution time and rate of errors. On a second attempt, all the participants tended to improve their results in doing these surgical tasks; nevertheless, after subgroup analysis of the T1 results, the S group had a better improvement of both parameters. Our laparoscopic simulator is simple to build, low-cost, easy to use, and seems to be a suitable resource for improving laparoscopic skills. In the future, further studies should evaluate the potential of this laparoscopic box on long-term surgical training with more complex tasks and simulation attempts.

Objective classification of psychomotor laparoscopic skills of surgeons based on three different approaches

BACKGROUND

The determination of surgeons' psychomotor skills in minimally invasive surgery techniques is one of the major concerns of the programs of surgical training in several hospitals. Therefore, it is important to assess and classify objectively the level of experience of surgeons and residents during their training process. The aim of this study was to investigate three classification methods for establishing automatically the level of surgical competence of the surgeons based on their psychomotor laparoscopic skills.

METHODS

A total of 43 participants, divided into an experienced surgeons group with ten experts (> 100 laparoscopic procedures performed) and non-experienced surgeons group with 24 residents and nine medical students (< 10 laparoscopic procedures performed), performed three tasks in the EndoViS training system. Motion data of the instruments were captured with a video-tracking system built into the EndoViS simulator and analyzed using 13 motion analysis parameters (MAPs). Radial basis function networks (RBFNets), K-star (K*), and random forest (RF) were used for classifying surgeons based on the MAPs' scores of all participants. The performance of the three classifiers was examined using hold-out and leave-one-out validation techniques.

RESULTS

For all three tasks, the K-star method was superior in terms of accuracy and AUC in both validation techniques. The mean accuracy of the classifiers was 93.33% for K-star, 87.58% for RBFNets, and 84.85% for RF in hold-out validation, and 91.47% for K-star, 89.92% for RBFNets, and 83.72% for RF in leave-one-out cross-validation.

CONCLUSIONS

The three proposed methods demonstrated high performance in the classification of laparoscopic surgeons, according to their level of psychomotor skills. Together with motion analysis and three laparoscopic tasks of the Fundamental Laparoscopic Surgery Program, these classifiers provide a means for objectively classifying surgical competence of the surgeons for existing laparoscopic box trainers.

Construct validity of eoSim - a low-cost and portable laparoscopic simulator

Purpose: To examine the construct validity of the low-cost, portable laparoscopic simulator eoSim using motion analysis.Material and methods: Novice and experienced surgeons (≤ 100 and >100 laparoscopic procedures performed, respectively) completed four tasks on the eoSim using the SurgTrac software: intracorporeal suture and tie, tube ligation, peg capping and precision cutting. The following metrics were recorded: Time to complete task, distance traveled, handedness (left- versus right hand use), time off-screen, distance between instrument tips, speed, acceleration and motion smoothness.Results: Compared to novices (n = 22), experienced surgeons (n = 14) completed tasks in less time (p ≤ .025), except when performing peg capping (p = .052). On all tasks, they also scored lower on the distance metric (p ≤ .001). Differences in handedness (left hand compared between groups, right hand compared between groups) were found to be significant for three tasks (p ≤ .025). In general, the experienced group made greater use of their left hand than the novice group.Conclusion: The eoSim can differentiate between experienced and novice surgeons on the tasks intracorporeal suture and tie, tube ligation and precision cutting, thus providing a convenient method for surgical departments to implement testing of their surgeons' basic laparoscopic skills.

Simulation platforms to assess laparoscopic suturing skills: a scoping review

BACKGROUND

Laparoscopic suturing (LS) has become a common technique used in a variety of advanced laparoscopic procedures. However, LS is a challenging skill to master, and many trainees may not be competent in performing LS at the end of their training. The purpose of this review is to identify simulation platforms available for assessment of LS skills, and determine the characteristics of the platforms and the LS skills that are targeted.

METHODS

A scoping review was conducted between January 1997 and October 2018 for full-text articles. The search was done in various databases. Only articles written in English or French were included. Additional studies were identified through reference lists. The search terms included "laparoscopic suturing" and "clinical competence."

RESULTS

Sixty-two studies were selected. The majority of the simulation platforms were box trainers with inanimate tissue, and targeted basic suturing and intracorporeal knot-tying techniques. Most of the validation came from internal structure (rater reliability) and relationship to other variables (compare training levels/case experience, and various metrics). Consequences were not addressed in any of the studies.

CONCLUSION

We identified many types of simulation platforms that were used for assessing LS skills, with most being for assessment of basic skills. Platforms assessing the competence of trainees for advanced LS skills were limited. Therefore, future research should focus on development of LS tasks that better reflect the needs of the trainees.

Validation of a virtual reality laparoscopic appendicectomy simulator: a novel process using cognitive task analysis

BACKGROUND

Virtual reality (VR) simulation is a vital component of surgical training with demonstrated improvements in surgical quality and clinical outcome.

AIMS

To validate the LAP Mentor (Simbionix™) laparoscopic appendicectomy (LA) VR simulator with inclusion of a novel tool, Cognitive Task Analysis (CTA).

METHODS

Thirty-two novices and nine experienced surgeons performed two simulated LAs. An expert-consensus questionnaire guided face validity assessment. Content validity was assessed using CTA-derived questions encompassing eight operative steps and four decision points. Construct validity was evaluated using dexterity metrics, masked assessment of surgical quality using the OSATS global rating scale, and mental workload from two validated tools: the NASA-TLX and SMEQ. Ten novices performed eight further LAs for learning curve assessment.

RESULTS

Face validity was demonstrated across all domains. Considering content validity, the essential technical and non-technical steps were evident. The experienced group performed the procedure quicker (median time 361 vs. 538 s, P = 0.0039) with fewer total movements (426 vs. 641, P < 0.0001) and shorter idle time (131 vs. 199 s, P = 0.0006). This correlated with higher OSATS scores (median 33.5 vs. 22.2, P < 0.0001) and lower mental demand (NASA-TLX: 9.0 vs. 13.75, P = 0.012; SMEQ: 60 vs. 80, P = 0.0025), indicating construct validity. Learning curve data showed statistically significant improvements after the 7th session for procedure time, total movements and idle time, which correlated with reduction in mental demand.

CONCLUSIONS

The LAP Mentor demonstrates face, content and construct validity for LA; thus, it can be used as an effective tool in surgical training. Task repetition leads to achievement of expert benchmarks.

Interpretation of motion analysis of laparoscopic instruments based on principal component analysis in box trainer settings

BACKGROUND

Motion analysis parameters (MAPs) have been extensively validated for assessment of minimally invasive surgical skills. However, there are discrepancies on how specific MAPs, tasks, and skills match with each other, reflecting that motion analysis cannot be generalized independently of the learning outcomes of a task. Additionally, there is a lack of knowledge on the meaning of motion analysis in terms of surgical skills, making difficult the provision of meaningful, didactic feedback. In this study, new higher significance MAPs (HSMAPs) are proposed, validated, and discussed for the assessment of technical skills in box trainers, based on principal component analysis (PCA).

METHODS

Motion analysis data were collected from 25 volunteers performing three box trainer tasks (peg grasping/PG, pattern cutting/PC, knot suturing/KS) using the EVA tracking system. PCA was applied on 10 MAPs for each task and hand. Principal components were trimmed to those accounting for an explained variance > 80% to define the HSMAPs. Individual contributions of MAPs to HSMAPs were obtained by loading analysis and varimax rotation. Construct validity of the new HSMAPs was carried out at two levels of experience based on number of surgeries.

RESULTS

Three new HSMAPs per hand were defined for PG and PC tasks, and two per hand for KS task. PG presented validity for HSMAPs related to insecurity and economy of space. PC showed validity for HSMAPs related to cutting efficacy, peripheral unawareness, and confidence. Finally, KS presented validity for HSMAPs related with economy of space and knotting security.

CONCLUSIONS

PCA-defined HSMAPs can be used for technical skills' assessment. Construct validation and expert knowledge can be combined to infer how competences are acquired in box trainer tasks. These findings can be exploited to provide residents with meaningful feedback on performance. Future works will compare the new HSMAPs with valid scoring systems such as GOALS.

A randomized control trial to evaluate the importance of pre-training basic laparoscopic psychomotor skills upon the learning curve of laparoscopic intra-corporeal knot tying

Background

Training of basic laparoscopic psychomotor skills improves the acquisition of more advanced laparoscopic tasks, such as laparoscopic intra-corporeal knot tying (LICK). This randomized controlled trial was designed to evaluate whether pre-training of basic skills, as laparoscopic camera navigation (LCN), hand-eye coordination (HEC), and bimanual coordination (BMC), and the combination of the three of them, has any beneficial effect upon the learning curve of LICK. The study was carried out in a private center in Asunción, Paraguay, by 80 medical students without any experience in surgery. Four laparoscopic tasks were performed in the ENCILAP model (LCN, HEC, BMC, and LICK). Participants were allocated to 5 groups (G1-G5). The study was structured in 5 phases. In phase 1, they underwent a base-line test (T₁) for all tasks (1 repetition of each task in consecutive order). In phase 2, participants underwent different training programs (30 consecutive repetitions) for basic tasks according to the group they belong to (G1: none; G2: LCN; G3: HEC; G4: BMC; and G5: LCN, HEC, and BMC). In phase 3, they were tested again (T₂) in the same manner than at T₁. In phase 4, they underwent a standardized training program for LICK (30 consecutive repetitions). In phase 5, they were tested again (T₃) in the same manner than at T₁ and T₂. At each repetition, scoring was based on the time taken for task completion system.

Results

The scores were plotted and non-linear regression models were used to fit the learning curves to one- and two-phase exponential decay models for each participant (individual curves) and for each group (group curves). The LICK group learning curves fitted better to the two-phase exponential decay model. From these curves, the starting points (Y0), the point after HEC training/before LICK training (Y1), the Plateau, and the rate constants (K) were calculated. All groups, except for G4, started from a similar point (Y0). At Y1, G5 scored already better than the others (G1 p = .004; G2 p = .04; G3 p < .0001; G4 NS). Although all groups reached a similar Plateau, G5 has a quicker learning than the others, demonstrated by a higher K (G1 p < 0.0001; G2 p < 0.0001; G3 p < 0.0001; and G4 p < 0.0001).

Conclusions

Our data confirms that training improves laparoscopic skills and demonstrates that pre-training of all basic skills (i.e., LCN, HEC, and BMC) shortens the LICK learning curve.

Dominant hand, non-dominant hand, or both? The effect of pre-training in hand-eye coordination upon the learning curve of laparoscopic intra-corporeal knot tying

Background

Training of basic laparoscopic psychomotor skills improves both acquisition and retention of more advanced laparoscopic tasks, such as laparoscopic intra-corporeal knot tying (LICK). This randomized controlled trial (RCT) was performed to evaluate the effect of different pre-training programs in hand-eye coordination (HEC) upon the learning curve of LICK.

Results

The study was performed in a private center in Asunción, Paraguay, by 60 residents/specialists in gynaecology with no experience in laparoscopic surgery. Participants were allocated in three groups. In phase ₁, a baseline test was performed (T₁, three repetitions). In phase 2, participants underwent different training programs for HEC (60 repetitions): G1 with both the dominant hand (DH) and the non-dominant hand (NDH), G2 with the DH only, G3 none. In phase 3, a post HEC/pre LICK training test was performed (T₂, three repetitions). In phase 4, participants underwent a standardized training program for LICK (60 repetitions). In phase 5, a final test was performed (T₃, three repetitions). The score was based on the time taken for task completion system. The scores were plotted and non-linear regression models were used to fit the learning curves to one- and two-phase exponential decay models for each participant (individual curves) and for each group (group curves). For both HEC and LICK, the group learning curves fitted better to the two-phase exponential decay model. For HEC with the DH, G1 and G2 started from a similar point, but G1 reached a lower plateau at a higher speed. In G1, the DH curve started from a lower point than the NDH curve, but both curves reached a similar plateau at comparable speeds. For LICK, all groups started from a similar point, but immediately after HEC training and before LICK training, G1 scored better than the others. All groups reached a similar plateau but with a different decay, G1 reaching this plateau faster than the others groups.

Conclusions

This study demonstrates that pre-training in HEC with both the DH and the NDH shortens the LICK learning curve.

Objective Assessment of Laparoscopic Force and Psychomotor Skills in a Novel Virtual Reality-Based Haptic Simulator

BACKGROUND

Most of the commercially available virtual reality-based laparoscopic simulators do not effectively evaluate combined psychomotor and force-based laparoscopic skills. Consequently, the lack of training on these critical skills leads to intraoperative errors.

OBJECTIVES

To assess the effectiveness of the novel virtual reality-based simulator, this study analyzed the combined psychomotor (i.e., motion or movement) and force skills of residents and expert surgeons. The study also examined the effectiveness of real-time visual force feedback and tool motion during training.

DESIGN

Bimanual fundamental (i.e., probing, pulling, sweeping, grasping, and twisting) and complex tasks (i.e., tissue dissection) were evaluated. In both tasks, visual feedback on applied force and tool motion were provided. The skills of the participants while performing the early tasks were assessed with and without visual feedback. Participants performed 5 repetitions of fundamental and complex tasks. Reaction force and instrument acceleration were used as metrics.

SETTING

Surgical Gastroenterology, Government Stanley Medical College and Hospital; Institute of Surgical Gastroenterology, Madras Medical College and Rajiv Gandhi Government General Hospital.

PARTICIPANTS

Residents (N = 25; postgraduates and surgeons with <2 years of laparoscopic surgery) and expert surgeons (N = 25; surgeons with >4 and ≤10 years of laparoscopic surgery).

RESULTS

Residents applied large forces compared with expert surgeons and performed abrupt tool movements (p < 0.001). However, visual + haptic feedback improved the performance of residents (p < 0.001). In complex tasks, visual + haptic feedback did not influence the applied force of expert surgeons, but influenced their tool motion (p < 0.001). Furthermore, in complex tissue sweeping task, expert surgeons applied more force, but were within the tissue damage limits. In both groups, exertion of large forces and abrupt tool motion were observed during grasping, probing or pulling, and tissue sweeping maneuvers (p < 0.001).

CONCLUSIONS

Modern day curriculum-based training should evaluate the skills of residents with robust force and psychomotor-based exercises for proficient laparoscopy. Visual feedback on force and motion during training has the potential to enhance the learning curve of residents.