A decade of imaging surgeons' brain function (part II): A systematic review of applications for technical and nontechnical skills assessment

Using neuroimaging to assess brain activity and areas associated with surgical skills: a systematic review

BACKGROUND

Surgical skills acquisition is under continuous development due to the emergence of new technologies, and there is a need for assessment tools to develop along with these. A range of neuroimaging modalities has been used to map the functional activation of brain networks while surgeons acquire novel surgical skills. These have been proposed as a method to provide a deeper understanding of surgical expertise and offer new possibilities for the personalized training of future surgeons. With studies differing in modalities, outcomes, and surgical skills there is a need for a systematic review of the evidence. This systematic review aims to summarize the current knowledge on the topic and evaluate the potential use of neuroimaging in surgical education.

METHODS

We conducted a systematic review of neuroimaging studies that mapped functional brain activation while surgeons with different levels of expertise learned and performed technical and non-technical surgical tasks. We included all studies published before July 1st, 2023, in MEDLINE, EMBASE and WEB OF SCIENCE.

RESULTS

38 task-based brain mapping studies were identified, consisting of randomized controlled trials, case-control studies, and observational cohort or cross-sectional studies. The studies employed a wide range of brain mapping modalities, including electroencephalography, functional magnetic resonance imaging, positron emission tomography, and functional near-infrared spectroscopy, activating brain areas involved in the execution and sensorimotor or cognitive control of surgical skills, especially the prefrontal cortex, supplementary motor area, and primary motor area, showing significant changes between novices and experts.

CONCLUSION

Functional neuroimaging can reveal how task-related brain activity reflects technical and non-technical surgical skills. The existing body of work highlights the potential of neuroimaging to link task-related brain activity patterns with the individual level of competency or improvement in performance after training surgical skills. More research is needed to establish its validity and usefulness as an assessment tool.

Optical neuroimaging and neurostimulation in surgical training and assessment: A state-of-the-art review

Introduction

Functional near-infrared spectroscopy (fNIRS) is a non-invasive optical neuroimaging technique used to assess surgeons' brain function. The aim of this narrative review is to outline the effect of expertise, stress, surgical technology, and neurostimulation on surgeons' neural activation patterns, and highlight key progress areas required in surgical neuroergonomics to modulate training and performance.

Methods

A literature search of PubMed and Embase was conducted to identify neuroimaging studies using fNIRS and neurostimulation in surgeons performing simulated tasks.

Results

Novice surgeons exhibit greater haemodynamic responses across the pre-frontal cortex than experts during simple surgical tasks, whilst expert surgical performance is characterized by relative prefrontal attenuation and upregulation of activation foci across other regions such as the supplementary motor area. The association between PFC activation and mental workload follows an inverted-U shaped curve, activation increasing then attenuating past a critical inflection point at which demands outstrip cognitive capacity Neuroimages are sensitive to the impact of laparoscopic and robotic tools on cognitive workload, helping inform the development of training programs which target neural learning curves. FNIRS differs in comparison to current tools to assess proficiency by depicting a cognitive state during surgery, enabling the development of cognitive benchmarks of expertise. Finally, neurostimulation using transcranial direct-current-stimulation may accelerate skill acquisition and enhance technical performance.

Conclusion

FNIRS can inform the development of surgical training programs which modulate stress responses, cognitive learning curves, and motor skill performance. Improved data processing with machine learning offers the possibility of live feedback regarding surgeons' cognitive states during operative procedures.

Electroencephalography can provide advance warning of technical errors during laparoscopic surgery

BACKGROUND

Intraoperative adverse events lead to patient injury and death, and are increasing. Early warning systems (EWSs) have been used to detect patient deterioration and save lives. However, few studies have used EWSs to monitor surgical performance and caution about imminent technical errors. Previous (non-surgical) research has investigated neural activity to predict future motor errors using electroencephalography (EEG). The present proof-of-concept cohort study investigates whether EEG could predict technical errors in surgery.

METHODS

In a large academic hospital, three surgical fellows performed 12 elective laparoscopic general surgeries. Audiovisual data of the operating room and the surgeon's neural activity were recorded. Technical errors and epochs of good surgical performance were coded into events. Neural activity was observed 40 s prior and 10 s after errors and good events to determine how far in advance errors were detected. A hierarchical regression model was used to account for possible clustering within surgeons. This prospective, proof-of-concept, cohort study was conducted from July to November 2021, with a pilot period from February to March 2020 used to optimize the technique of data capture and included participants who were blinded from study hypotheses.

RESULTS

Forty-five technical errors, mainly due to too little force or distance (n = 39), and 27 good surgical events were coded during grasping and dissection. Neural activity representing error monitoring (p = .008) and motor uncertainty (p = .034) was detected 17 s prior to errors, but not prior to good surgical performance.

CONCLUSIONS

These results show that distinct neural signatures are predictive of technical error in laparoscopic surgery. If replicated with low false-alarm rates, an EEG-based EWS of technical errors could be used to improve individualized surgical training by flagging imminent unsafe actions-before errors occur and cause patient harm.

Use of neuroimaging to measure neurocognitive engagement in health professions education: a scoping review

PURPOSE

To map the current literature on functional neuroimaging use in medical education research as a novel measurement modality for neurocognitive engagement, learning, and expertise development.

METHOD

We searched PubMed, Embase, Cochrane, ERIC, and Web of Science, and hand-searched reference lists of relevant articles on April 4, 2019, and updated the search on July 7, 2020. Two authors screened the abstracts and then full-text articles for eligibility based on inclusion criteria. The data were then charted, synthesized, and analyzed descriptively.

RESULTS

Sixty-seven articles published between 2007 and 2020 were included in this scoping review. These studies used three main neuroimaging modalities: functional magnetic resonance imaging, functional near-infrared spectroscopy, and electroencephalography. Most of the publications (90%, n = 60) were from the last 10 years (2011-2020). Although these studies were conducted in 16 countries, 68.7% (n = 46) were from three countries: the USA (n = 21), UK (n = 15), and Canada (n = 10). These studies were mainly non-experimental (74.6%, n = 50). Most used neuroimaging techniques to examine psychomotor skill development (57%, n = 38), but several investigated neurocognitive correlates of clinical reasoning skills (22%, n = 15).

CONCLUSION

This scoping review maps the available literature on functional neuroimaging use in medical education. Despite the heterogeneity in research questions, study designs, and outcome measures, we identified a few common themes. Included studies are encouraging of the potential for neuroimaging to complement commonly used measures in education research and may help validate/challenge established theoretical assumptions and provide insight into training methods. This review highlighted several areas for further research. The use of these emerging technologies appears ripe for developing precision education, establishing viable study protocols for realistic operational settings, examining team dynamics, and exploring applications for real-time monitoring/intervention during critical clinical tasks.

Methods and Measures for Mental Stress Assessment in Surgery: A Systematic Review of 20 Years of Literature

Real-time mental stress monitoring from surgeons and surgical staff in operating rooms may reduce surgical injuries, improve performance and quality of medical care, and accelerate implementation of stress-management strategies. Motivated by the increase in usage of objective and subjective metrics for cognitive monitoring and by the gap in reviews of experimental design setups and data analytics, a systematic review of 71 studies on mental stress and workload measurement in surgical settings, published in 2001-2020, is presented. Almost 61% of selected papers used both objective and subjective measures, followed by 25% that only administered subjective tools - mostly consisting of validated instruments and customized surveys. An overall increase in the total number of publications on intraoperative stress assessment was observed from mid-2010 s along with a momentum in the use of both subjective and real-time objective measures. Cardiac activity, including heart-rate variability metrics, stress hormones, and eye-tracking metrics were the most frequently and electroencephalography (EEG) was the least frequently used objective measures. Around 40% of selected papers collected at least two objective measures, 41% used wearable devices, 23% performed synchronization and annotation, and 76% conducted baseline or multi-point data acquisition. Furthermore, 93% used a variety of statistical techniques, 14% applied regression models, and only one study released a public, anonymized dataset. This review of data modalities, experimental setups, and analysis techniques for intraoperative stress monitoring highlights the initiatives of surgical data science and motivates research on computational techniques for mental and surgical skills assessment and cognition-guided surgery.

Neuromonitoring Correlates of Expertise Level in Surgical Performers: A Systematic Review

Surgical expertise does not have a clear definition and is often culturally associated with power, authority, prestige, and case number rather than more objective proxies of excellence. Multiple models of expertise progression have been proposed including the Dreyfus model, however, they all currently require subjective evaluation of skill. Recently, efforts have been made to improve the ways in which surgical excellence is measured and expertise is defined using artificial intelligence, video recordings, and accelerometers. However, these aforementioned methods of assessment are still subjective or indirect proxies of expertise, thus uncovering the neural mechanisms that differentiate expert surgeons from trainees may enhance the objectivity of surgical expertise validation. In fact, some researchers have already suggested that their neural imaging-based expertise classification methods outperform currently used methods of surgical skill certification such as the Fundamentals of Laparoscopic Surgery (FLS) scores. Such imaging biomarkers would not only help better identify the highest performing surgeons, but could also improve residency programs by providing more objective, evidence-based feedback and developmental milestones for those in training and perhaps act as a marker of surgical potential in medical students. Despite the potential advantages of using neural imaging in the assessment of surgical expertise, this field of research remains in its infancy. This systematic review identifies studies that have applied neuromonitoring in assessing surgical skill across levels of expertise. The goals of this review are to identify (1) the strongest neural indicators of surgical expertise, (2) the limitations of the current literature on this subject, (3) the most sensible future directions for further study. We found substantial evidence that surgical expertise can be delineated by differential activation and connectivity in the prefrontal cortex (PFC) across multiple task and neuroimaging modalities. Specifically, novices tend to have greater PFC activation than experts under standard conditions in bimanual and decision-making tasks. However, under high temporal demand tasks, experts had increased PFC activation whereas novices had decreased PFC activation. Common limitations uncovered in this review were that task difficulty was often insufficient to delineate between residents and attending. Moreover, attending level involvement was also low in multiple studies which may also have contributed to this issue. Most studies did not analyze the ability of their neuromonitoring findings to accurately classify subjects by level of expertise. Finally, the predominance of fNIRS as the neuromonitoring modality limits our ability to uncover the neural correlates of surgical expertise in non-cortical brain regions. Future studies should first strive to address these limitations. In the longer term, longitudinal within-subjects design over the course of a residency or even a career will also advance the field. Although logistically arduous, such studies would likely be most beneficial in demonstrating effects of increasing surgical expertise on regional brain activation and inter-region connectivity.

Dataset of prefrontal transcranial direct-current stimulation to improve early surgical knot-tying skills

Transcranial direct-current stimulation (tDCS) has previously demonstrated promising effects in improving surgical performance with motor region stimulation [1], [2], [3], [4]. However, extensive prior research has revealed an important role of the prefrontal cortex in surgical skill development [5,6]. This article presents the data of a double-blind randomized sham-controlled trial investigating the effect of prefrontal tDCS on knot-tying performance [7]. Data was collected from an active (n = 20) and sham (n = 20) group across three blocks: pre-, online- (during) and post-tDCS. Group and block differences of knot-tying performance were analyzed using a Generalized linear mixed model and supported with a Friedman's test. Further sub-analyses were conducted to compare high vs. low skilled individuals and initial vs. last knots. Subjective workload was assessed after each block using a SURG-TLX questionnaire and side-effects of the tDCS block were recorded using an additional survey.

High density optical neuroimaging predicts surgeons's subjective experience and skill levels

Measuring cognitive load is important for surgical education and patient safety. Traditional approaches of measuring cognitive load of surgeons utilise behavioural metrics to measure performance and surveys and questionnaires to collect reports of subjective experience. These have disadvantages such as sporadic data, occasionally intrusive methodologies, subjective or misleading self-reporting. In addition, traditional approaches use subjective metrics that cannot distinguish between skill levels. Functional neuroimaging data was collected using a high density, wireless NIRS device from sixteen surgeons (11 attending surgeons and 5 surgery resident) and 17 students while they performed two laparoscopic tasks (Peg transfer and String pass). Participant’s subjective mental load was assessed using the NASA-TLX survey. Machine learning approaches were used for predicting the subjective experience and skill levels. The Prefrontal cortex (PFC) activations were greater in students who reported higher-than-median task load, as measured by the NASA-TLX survey. However in the case of attending surgeons the opposite tendency was observed, namely higher activations in the lower v higher task loaded subjects. We found that response was greater in the left PFC of students particularly near the dorso- and ventrolateral areas. We quantified the ability of PFC activation to predict the differences in skill and task load using machine learning while focussing on the effects of NIRS channel separation distance on the results. Our results showed that the classification of skill level and subjective task load could be predicted based on PFC activation with an accuracy of nearly 90%. Our finding shows that there is sufficient information available in the optical signals to make accurate predictions about the surgeons’ subjective experiences and skill levels. The high accuracy of results is encouraging and suggest the integration of the strategy developed in this study as a promising approach to design automated, more accurate and objective evaluation methods.

Frontal theta brain activity varies as a function of surgical experience and task error

OBJECTIVE

Investigations into surgical expertise have almost exclusively focused on overt behavioral characteristics with little consideration of the underlying neural processes. Recent advances in neuroimaging technologies, for example, wireless, wearable scalp-recorded electroencephalography (EEG), allow an insight into the neural processes governing performance. We used scalp-recorded EEG to examine whether surgical expertise and task performance could be differentiated according to an oscillatory brain activity signal known as frontal theta-a putative biomarker for cognitive control processes.

DESIGN SETTING AND PARTICIPANTS

Behavioral and EEG data were acquired from dental surgery trainees with 1 year (n=25) and 4 years of experience (n=20) while they performed low and high difficulty drilling tasks on a virtual reality surgical simulator. EEG power in the 4-7 Hz range in frontal electrodes (indexing frontal theta) was examined as a function of experience, task difficulty and error rate.

RESULTS

Frontal theta power was greater for novices relative to experts (p=0.001), but did not vary according to task difficulty (p=0.15) and there was no Experience × Difficulty interaction (p=0.87). Brain-behavior correlations revealed a significant negative relationship between frontal theta and error in the experienced group for the difficult task (r=-0.594, p=0.0058), but no such relationship emerged for novices.

CONCLUSION

We find frontal theta power differentiates between surgical experiences but correlates only with error rates for experienced surgeons while performing difficult tasks. These results provide a novel perspective on the relationship between expertise and surgical performance.

Use of prefrontal cortex activity as a measure of learning curve in surgical novices: results of a single blind randomised controlled trial

BACKGROUND

Neurobiological feedback in surgical training could translate to better educational outcomes such as measures of learning curve. This work examined the variation in brain activation of medical students when performing laparoscopic tasks before and after a training workshop, using functional near-infrared spectroscopy (fNIRS).

METHODS AND PROCEDURES

This single blind randomised controlled trial examined the prefrontal cortex activity (PFCA) differences in two groups of novice medical students during the acquisition of four laparoscopic tasks. Both groups were shown a basic tutorial video, with the "Trained-group" receiving an additional standardised one-to-one training on the tasks. The PFCA was measured pre- and post-intervention using a portable fNIRS device and reported as mean total oxygenated hemoglobin (HbOµm). Primary outcome of the study is the difference in HbOµm between post- and pre-intervention readings for each of the four laparoscopic tasks. The pre- and post-intervention laparoscopic tasks were recorded and assessed by two blinded individual assessors for objective scores of the performance.

RESULTS

16 Trained and 16 Untrained, right-handed medical students with an equal sex distribution and comparable age distribution were recruited. Trained group had an attenuated left PFCA in the "Precision cutting" (p = 0.007) task compared to the Untrained group. Subgroup analysis by sex revealed attenuation in left PFCA in Trained females compared to Untrained females across two laparoscopic tasks: "Peg transfer" (p = 0.005) and "Precision cutting" (p = 0.003). No significant PFCA attenuation was found in male students who underwent training compared to Untrained males.

CONCLUSION

A standardised laparoscopic training workshop promoted greater PFCA attenuation in female medical students compared to males. This suggests that female and male students respond differently to the same instructional approach. Implications include a greater focus on one-to-one surgical training for female students and use of PFCA attenuation as a form of neurobiological feedback in surgical training.

Brain activation during laparoscopic tasks in high- and low-performing medical students: a pilot fMRI study

BACKGROUND

Up to 20% of medical students are unable to reach competency in laparoscopic surgery. It is unknown whether these difficulties arise from heterogeneity in neurological functioning across individuals. We sought to examine the differences in neurological functioning during laparoscopic tasks between high- and low-performing medical students using functional magnetic resonance imaging (fMRI).

METHODS

This prospective cohort study enrolled North American medical students who were within the top 20% and bottom 20% of laparoscopic performers from a previous study. Brain activation was recorded using fMRI while participants performed peg-pointing, intracorporeal knot tying (IKT), and the Pictorial Surface Orientation (PicSOr) test. Brain activation maps were created and areas of activation were compared between groups.

RESULTS

In total, 9/12 high and 9/13 low performers completed the study. High performers completed IKT faster and made more successful knot ties than low performers [standing: 23.5 (5.0) sec vs. 37.6 (18.4) sec, p = 0.03; supine: 23.2 (2.5) sec vs. 72.7 (62.8) sec, p = 0.02; number of successful ties supine, 3 ties vs. 1 tie, p = 0.01]. Low performers showed more brain activation than high performers in the peg-pointing task (q < 0.01), with no activation differences in the IKT task. There were no behavioral differences in the PiCSOr task.

CONCLUSIONS

This study is the first to show differences between low and high performers of laparoscopic tasks at the brain level. This pilot study has shown the feasibility of using fMRI to examine laparoscopic surgical skills. Future studies are needed for further exploration of our initial findings.

Association of Residents' Neural Signatures With Stress Resilience During Surgery

Importance

Intraoperative stressors may compound cognitive load, prompting performance decline and threatening patient safety. However, not all surgeons cope equally well with stress, and the disparity between performance stability and decline under high cognitive demand may be characterized by differences in activation within brain areas associated with attention and concentration such as the prefrontal cortex (PFC).

Objective

To compare PFC activation between surgeons demonstrating stable performance under temporal stress with those exhibiting stress-related performance decline.

Design, Setting, and Participants

Cohort study conducted from July 2015 to September 2016 at the Imperial College Healthcare National Health Service Trust, England. One hundred two surgical residents (postgraduate year 1 and greater) were invited to participate, of which 33 agreed to partake.

Exposures

Participants performed a laparoscopic suturing task under 2 conditions: self-paced (SP; without time-per-knot restrictions), and time pressure (TP; 2-minute per knot time restriction).

Main Outcomes and Measures

A composite deterioration score was computed based on between-condition differences in task performance metrics (task progression score [arbitrary units], error score [millimeters], leak volume [milliliters], and knot tensile strength [newtons]). Based on the composite score, quartiles were computed reflecting performance stability (quartile 1 [Q1]) and decline (quartile 4 [Q4]). Changes in PFC oxygenated hemoglobin concentration (HbO2) measured at 24 different locations using functional near-infrared spectroscopy were compared between Q1 and Q4. Secondary outcomes included subjective workload (Surgical Task Load Index) and heart rate.

Results

Of the 33 participants, the median age was 33 years, the range was 29 to 56 years, and 27 were men (82%). The Q1 residents demonstrated task-induced increases in HbO2 across the bilateral ventrolateral PFC (VLPFC) and right dorsolateral PFC in the SP condition and in the VLPFC in the TP condition. In contrast, Q4 residents demonstrated decreases in HbO2 in both conditions. The magnitude of PFC activation (change in HbO2) was significantly greater in Q1 than Q4 across the bilateral VLPFC during both SP (mean [SD] left VLPFC: Q1, 0.44 [1.30] μM; Q4, -0.21 [2.05] μM; P < .001; right VLPFC: Q1, 0.46 [1.12] μM; Q4, -0.15 [2.14] μM; P < .001) and TP (mean [SD] left VLPFC: Q1, 0.44 [1.36] μM; Q4, -0.03 [1.83] μM; P = .001; right VLPFC: Q1, 0.49 [1.70] μM; Q4, -0.32 [2.00] μM; P < .001) conditions. There were no significant between-group differences in Surgical Task Load Index or heart rate in either condition.

Conclusions and Relevance

Performance stability within TP is associated with sustained prefrontal activation indicative of preserved attention and concentration, whereas performance decline is associated with prefrontal deactivation that may represent task disengagement.

Robotic Surgery Improves Technical Performance and Enhances Prefrontal Activation During High Temporal Demand

Robotic surgery may improve technical performance and reduce mental demands compared to laparoscopic surgery. However, no studies have directly compared the impact of robotic and laparoscopic techniques on surgeons' brain function. This study aimed to assess the effect of the operative platform (robotic surgery or conventional laparoscopy) on prefrontal cortical activation during a suturing task performed under temporal demand. Eight surgeons (mean age ± SD = 34.5 ± 2.9 years, male:female ratio = 7:1) performed an intracorporeal suturing task in a self-paced manner and under a 2 min time restriction using conventional laparoscopic and robotic techniques. Prefrontal activation was assessed using near-infrared spectroscopy, subjective workload was captured using SURG-TLX questionnaires, and a continuous heart rate monitor measured systemic stress responses. Task progression scores (au), error scores (au), leak volumes (mL) and knot tensile strengths (N) provided objective assessment of technical performance. Under time pressure, robotic suturing led to improved technical performance (median task progression score: laparoscopic suturing = 4.5 vs. robotic suturing = 5.0; z = - 2.107, p = 0.035; median error score: laparoscopic suturing = 3.0 mm vs. robotic suturing = 2.1 mm; z = - 2.488, p = 0.013). Compared to laparoscopic suturing, greater prefrontal activation was identified in seven channels located primarily in lateral prefrontal regions. These results suggest that robotic surgery improves performance during high workload conditions and is associated with enhanced activation in regions of attention, concentration and task engagement.