Objective assessment of laparoscopic skills: dual-task approach

Neuromonitoring Correlates of Expertise Level in Surgical Performers: A Systematic Review

Surgical expertise does not have a clear definition and is often culturally associated with power, authority, prestige, and case number rather than more objective proxies of excellence. Multiple models of expertise progression have been proposed including the Dreyfus model, however, they all currently require subjective evaluation of skill. Recently, efforts have been made to improve the ways in which surgical excellence is measured and expertise is defined using artificial intelligence, video recordings, and accelerometers. However, these aforementioned methods of assessment are still subjective or indirect proxies of expertise, thus uncovering the neural mechanisms that differentiate expert surgeons from trainees may enhance the objectivity of surgical expertise validation. In fact, some researchers have already suggested that their neural imaging-based expertise classification methods outperform currently used methods of surgical skill certification such as the Fundamentals of Laparoscopic Surgery (FLS) scores. Such imaging biomarkers would not only help better identify the highest performing surgeons, but could also improve residency programs by providing more objective, evidence-based feedback and developmental milestones for those in training and perhaps act as a marker of surgical potential in medical students. Despite the potential advantages of using neural imaging in the assessment of surgical expertise, this field of research remains in its infancy. This systematic review identifies studies that have applied neuromonitoring in assessing surgical skill across levels of expertise. The goals of this review are to identify (1) the strongest neural indicators of surgical expertise, (2) the limitations of the current literature on this subject, (3) the most sensible future directions for further study. We found substantial evidence that surgical expertise can be delineated by differential activation and connectivity in the prefrontal cortex (PFC) across multiple task and neuroimaging modalities. Specifically, novices tend to have greater PFC activation than experts under standard conditions in bimanual and decision-making tasks. However, under high temporal demand tasks, experts had increased PFC activation whereas novices had decreased PFC activation. Common limitations uncovered in this review were that task difficulty was often insufficient to delineate between residents and attending. Moreover, attending level involvement was also low in multiple studies which may also have contributed to this issue. Most studies did not analyze the ability of their neuromonitoring findings to accurately classify subjects by level of expertise. Finally, the predominance of fNIRS as the neuromonitoring modality limits our ability to uncover the neural correlates of surgical expertise in non-cortical brain regions. Future studies should first strive to address these limitations. In the longer term, longitudinal within-subjects design over the course of a residency or even a career will also advance the field. Although logistically arduous, such studies would likely be most beneficial in demonstrating effects of increasing surgical expertise on regional brain activation and inter-region connectivity.

Distinguishing Intermediate and Novice Surgeons by Eye Movements

Surgical skill-level assessment is key to collecting the required feedback and adapting the educational programs accordingly. Currently, these assessments for the minimal invasive surgery programs are primarily based on subjective methods, and there is no consensus on skill level classifications. One of the most detailed of these classifications categorize skill levels as beginner, novice, intermediate, sub-expert, and expert. To properly integrate skill assessment into minimal invasive surgical education programs and provide skill-based training alternatives, it is necessary to classify the skill levels in as detailed a way as possible and identify the differences between all skill levels in an objective manner. Yet, despite the existence of very encouraging results in the literature, most of the studies have been conducted to better understand the differences between novice and expert surgical skill levels leaving out the other crucial skill levels between them. Additionally, there are very limited studies by considering the eye-movement behaviors of surgical residents. To this end, the present study attempted to distinguish novice- and intermediate-level surgical residents based on their eye movements. The eye-movement data was recorded from 23 volunteer surgical residents while they were performing four computer-based simulated surgical tasks under different hand conditions. The data was analyzed using logistic regression to estimate the skill levels of both groups. The best results of the estimation revealing a 91.3% recognition rate of predicting novice and intermediate surgical residents on one scenario were selected from four under the dominant hand condition. These results show that the eye-movements can be potentially used to identify surgeons with intermediate and novice skills. However, the results also indicate that the order in which the scenarios are provided, and the design of the scenario, the tasks, and their appropriateness with the skill levels of the participants are all critical factors to be considered in improving the estimation ratio, and hence require thorough assessment for future research.

Classification of Intermediate and Novice Surgeons' Skill Assessment Through Performance Metrics

Background. Endoscopic surgeries have become an alternative for open procedures whenever possible. For such types of operations, surgeons are required to gain several skills, whose development needs hands-on practice. Accordingly, gaining these skills today is a challenge for surgical education programs. Despite the development of several technology-enhanced training environments, there are still problems to better integrate these technologies into educational programs. For an appropriate integration, it is critical to assess the skill levels and adapt the training content according to the trainees' requirements. In the literature, there exist several methods for assessing these skill levels. However, there are still problems in practice for objective and repetitive assessment. Methods. The present study aims to estimate the skill levels of participants in surgical training programs in an objective manner by collecting experimental data from residents in an endoscopic surgical simulation environment and gathering performance metrics. Results. It is shown that, by comparing the results of a number of classification algorithms for the best accuracy estimation and feature set, the "novice" and "intermediate" skill levels can be estimated with an accuracy of 86%. Conclusions. The outcomes help surgical educators and instructional system designers to better assess the skill levels of the trainees and guide them accordingly. In addition, objective assessments as highlighted in this study can be beneficial when designing technology-enhanced adaptive learning environments.

Temporal Stress in the Operating Room: Brain Engagement Promotes "Coping" and Disengagement Prompts "Choking"

OBJECTIVE

To investigate the impact of time pressure (TP) on prefrontal activation and technical performance in surgical residents during a laparoscopic suturing task.

BACKGROUND

Neural mechanisms enabling surgeons to maintain performance and cope with operative stressors are unclear. The prefrontal cortex (PFC) is implicated due to its role in attention, concentration, and performance monitoring.

METHODS

A total of 33 residents [Postgraduate Year (PGY)1-2 = 15, PGY3-4 = 8, and PGY5 = 10] performed a laparoscopic suturing task under "self-paced" (SP) and "TP" conditions (TP = maximum 2 minutes per knot). Subjective workload was quantified using the Surgical Task Load Index. PFC activation was inferred using optical neuroimaging. Technical skill was assessed using progression scores (au), error scores (mm), leak volumes (mL), and knot tensile strengths (N).

RESULTS

TP led to greater perceived workload amongst all residents (mean Surgical Task Load Index score ± SD: PGY1-2: SP = 160.3 ± 24.8 vs TP = 202.1 ± 45.4, P < 0.001; PGY3-4: SP = 123.0 ± 52.0 vs TP = 172.5 ± 43.1, P < 0.01; PGY5: SP = 105.8 ± 55.3 vs TP = 159.1 ± 63.1, P < 0.05). Amongst PGY1-2 and PGY3-4, deterioration in task progression, error scores and knot tensile strength (P < 0.05), and diminished PFC activation was observed under TP. In PGY5, TP resulted in inferior task progression and error scores (P < 0.05), but preservation of knot tensile strength. Furthermore, PGY5 exhibited less attenuation of PFC activation under TP, and greater activation than either PGY1-2 or PGY3-4 under both experimental conditions (P < 0.05).

CONCLUSIONS

Senior residents cope better with temporal demands and exhibit greater technical performance stability under pressure, possibly due to sustained PFC activation and greater task engagement. Future work should seek to develop training strategies that recruit prefrontal resources, enhance task engagement, and improve performance under pressure.

Toward a Model of Human Information Processing for Decision-Making and Skill Acquisition in Laparoscopic Colorectal Surgery

OBJECTIVE

To create a human information-processing model for laparoscopic surgery based on already established literature and primary research to enhance laparoscopic surgical education in this context.

DESIGN

We reviewed the literature for information-processing models most relevant to laparoscopic surgery. Our review highlighted the necessity for a model that accounts for dynamic environments, perception, allocation of attention resources between the actions of both hands of an operator, and skill acquisition and retention. The results of the literature review were augmented through intraoperative observations of 7 colorectal surgical procedures, supported by laparoscopic video analysis of 12 colorectal procedures.

RESULTS

The Wickens human information-processing model was selected as the most relevant theoretical model to which we make adaptions for this specific application. We expanded the perception subsystem of the model to involve all aspects of perception during laparoscopic surgery. We extended the decision-making system to include dynamic decision-making to account for case/patient-specific and surgeon-specific deviations. The response subsystem now includes dual-task performance and nontechnical skills, such as intraoperative communication. The memory subsystem is expanded to include skill acquisition and retention.

CONCLUSIONS

Surgical decision-making during laparoscopic surgery is the result of a highly complex series of processes influenced not only by the operator's knowledge, but also patient anatomy and interaction with the surgical team. Newer developments in simulation-based education must focus on the theoretically supported elements and events that underpin skill acquisition and affect the cognitive abilities of novice surgeons. The proposed human information-processing model builds on established literature regarding information processing, accounting for a dynamic environment of laparoscopic surgery. This revised model may be used as a foundation for a model describing robotic surgery.

Tracking and discrete dual task performance for different visual spatial stimulus-response mappings with focal and ambient vision

The effect of spatial compatibility for various display-control configurations on human performance was studied with a dual-task paradigm using a tracking task and a discrete response task. Degradation of performance on both tasks within the visual modality was observed and was considered to be most likely due to resource competition resulting from simultaneous task operation. It was found that the more complicated the mapping for the discrete spatial compatibility response task, the more severe the interference with the tracking task. Although performance on both the tracking and spatial response tasks was impaired, the magnitude of impairment was not as great as expected, implying that focal and ambient vision required for the tracking task and spatial task, respectively, might be deployed, at least partly, from separate resources. Participants here seemed to successfully use focal vision for tracking and ambient vision for identifying signal lights concurrently, reducing the expected keen competition for visual resources.

The Impact of Distraction on Laparoscopic Skills in Veterinary Medical Students

OBJECTIVE

To investigate the impact of distraction on the performance of a simulator-based laparoscopic task in veterinary students.

STUDY DESIGN

Prospective, randomized trial.

SAMPLE POPULATION

Years 1-4 veterinary students (n=41).

METHODS

Participants repeated a simulated laparoscopic peg transfer task to eliminate any learning effects and were subsequently randomized to receive either a cognitive (double-digit addition questions, n=21) or sensory distraction (dogs barking and anesthesia monitor alerts, n=20). The laparoscopic task scores were compared between baseline and in the presence and absence of each distraction. The number of addition questions attempted, and the number of questions answered correctly in 1 minute were compared between baseline and during a concurrent laparoscopic task.

RESULTS

Baseline laparoscopic task scores were not significantly different between groups (P=.09). Laparoscopic task scores were significantly lower than baseline when performed with the cognitive distraction (P<.001) and significantly higher than baseline when performed with the sensory distraction (P=.005). Participants undergoing cognitive distraction attempted significantly fewer addition questions (P<.001) and answered significantly fewer addition questions correctly (P<.001) when a concurrent laparoscopic task was performed.

CONCLUSION

Cognitive distraction had a negative impact on the performance of a laparoscopic task in this cohort of veterinary students, whereas sensory distraction had a positive effect.

Detection of Changes in Surgical Difficulty

Background. Assessing the workload of surgeons requires technology to continuously monitor surgeons’ behaviors without interfering with their performance. We investigated the feasibility of using eye-tracking to reveal surgeons’ response to increasing task difficulty. Methods. A controlled study was conducted in a simulated operating room, where 14 subjects were required to perform a laparoscopic procedure that includes 9 subtasks. The subtasks could be divided into 3 types with different levels of task difficulty, calculated by the index of task difficulty (ID) proposed by Fitts in 1954. Pupillary responses of subjects in performing the procedure were recorded using Tobii eye-tracking equipment. Peak pupil dilation and movement time were compared between subtasks with different IDs as well as between fast moving and slow aiming phases within each subtask. Results. When the task difficulty was increased, task completion time increased. Meanwhile, the subjects’ peak pupil size also increased. As the entire procedure was performed continuously, we found that pupil responses were not only affected by the ID in the current subtask but also influenced by subtasks before and after. Discussion. Decomposing a surgical procedure into meaningful subtasks and examining the surgeon’s pupil response to each subtask enables us to identify the challenging steps within a continuous surgical procedure. Psychomotor evidence on surgeon’s performance may lead to an innovation for designing a task-specific training curriculum.

The effect of divided attention on novices and experts in laparoscopic task performance

BACKGROUND

Attention is important for the skilful execution of surgery. The surgeon's attention during surgery is divided between surgery and outside distractions. The effect of this divided attention has not been well studied previously. We aimed to compare the effect of dividing attention of novices and experts on a laparoscopic task performance.

METHODS

Following ethical approval, 25 novices and 9 expert surgeons performed a standardised peg transfer task in a laboratory setup under three randomly assigned conditions: silent as control condition and two standardised auditory distracting tasks requiring response (easy and difficult) as study conditions. Human reliability assessment was used for surgical task analysis. Primary outcome measures were correct auditory responses, task time, number of surgical errors and instrument movements. Secondary outcome measures included error rate, error probability and hand specific differences. Non-parametric statistics were used for data analysis.

RESULTS

21109 movements and 9036 total errors were analysed. Novices had increased mean task completion time (seconds) (171 ± 44SD vs. 149 ± 34, p < 0.05), number of total movements (227 ± 27 vs. 213 ± 26, p < 0.05) and number of errors (127 ± 51 vs. 96 ± 28, p < 0.05) during difficult study conditions compared to control. The correct responses to auditory stimuli were less frequent in experts (68 %) compared to novices (80 %). There was a positive correlation between error rate and error probability in novices (r (2) = 0.533, p < 0.05) but not in experts (r (2) = 0.346, p > 0.05).

CONCLUSION

Divided attention conditions in theatre environment require careful consideration during surgical training as the junior surgeons are less able to focus their attention during these conditions.