Objective assessment of surgical skill transfer using non-invasive brain imaging

Neuromechanisms of simulation-based arthroscopic skills assessment: a fNIRS study

BACKGROUND

The neural mechanisms underlying differences in the performance of simulated arthroscopic skills across various skill levels remain unclear. Our primary objective is to investigate the learning mechanisms of simulated arthroscopic skills using functional near-infrared spectroscopy (fNIRS).

METHODS

We recruited 27 participants, divided into three groups: novices (n = 9), intermediates (n = 9), and experts (n = 9). Participants completed seven arthroscopic tasks on a simulator, including diagnostic navigation, triangulation, grasping stars, diagnostic exploration, meniscectomy, synovial membrane cleaning, and loose body removal. All tasks were videotaped and assessed via the simulator system and the Arthroscopic Surgical Skill Evaluation Tool (ASSET), while cortical activation data were collected using fNIRS. Simulator scores and ASSET scores were analyzed to identify different level of performance of all participants. Brain region activation and functional connectivity (FC) of different types of participants were analyzed from fNIRS data.

RESULTS

Both the expert and intermediate groups scored significantly higher than the novice group (p < 0.001). There were significant differences in ASSET scores between experts and intermediates, experts and novices, and intermediates and novices (p = 0.0047, p < 0.0001, p < 0.0001), with the trend being experts > intermediates > novices. The intermediate group exhibited significantly greater activation in the left primary motor cortex (LPMC) and left prefrontal cortex (LPFC) compared to the novice group (p = 0.0152, p = 0.0021). Compared to experts, the intermediate group demonstrated significantly increased FC between the presupplementary motor area (preSMA) and the right prefrontal cortex (RPFC; p < 0.001). Additionally, the intermediate group showed significantly increased FC between the preSMA and LPFC, RPFC and LPFC, and LPMC and LPFC compared to novices (p = 0.0077, p = 0.0285, p = 0.0446).

CONCLUSION

Cortical activation and functional connectivity reveal varying levels of activation intensity in the PFC, PMC, and preSMA among novices, intermediates, and experts. The intermediate group exhibited the highest activation intensity.

Using neuroimaging to assess brain activity and areas associated with surgical skills: a systematic review

BACKGROUND

Surgical skills acquisition is under continuous development due to the emergence of new technologies, and there is a need for assessment tools to develop along with these. A range of neuroimaging modalities has been used to map the functional activation of brain networks while surgeons acquire novel surgical skills. These have been proposed as a method to provide a deeper understanding of surgical expertise and offer new possibilities for the personalized training of future surgeons. With studies differing in modalities, outcomes, and surgical skills there is a need for a systematic review of the evidence. This systematic review aims to summarize the current knowledge on the topic and evaluate the potential use of neuroimaging in surgical education.

METHODS

We conducted a systematic review of neuroimaging studies that mapped functional brain activation while surgeons with different levels of expertise learned and performed technical and non-technical surgical tasks. We included all studies published before July 1st, 2023, in MEDLINE, EMBASE and WEB OF SCIENCE.

RESULTS

38 task-based brain mapping studies were identified, consisting of randomized controlled trials, case-control studies, and observational cohort or cross-sectional studies. The studies employed a wide range of brain mapping modalities, including electroencephalography, functional magnetic resonance imaging, positron emission tomography, and functional near-infrared spectroscopy, activating brain areas involved in the execution and sensorimotor or cognitive control of surgical skills, especially the prefrontal cortex, supplementary motor area, and primary motor area, showing significant changes between novices and experts.

CONCLUSION

Functional neuroimaging can reveal how task-related brain activity reflects technical and non-technical surgical skills. The existing body of work highlights the potential of neuroimaging to link task-related brain activity patterns with the individual level of competency or improvement in performance after training surgical skills. More research is needed to establish its validity and usefulness as an assessment tool.

Assessment of learning in simulator-based arthroscopy training with the diagnostic arthroscopy skill score (DASS) and neurophysiological measures

PURPOSE

Virtual arthroscopic training has become increasingly popular. However, there is a lack of efficiency-based tracking of the trainee, which may be critical for determining the specifics of training programs and adapting them for the needs of each trainee. This study aims to evaluate and compare the measures obtained with a non-invasive neurophysiological method with The Diagnostic Arthroscopy Skill Score (DASS), a commonly used assessment tool for evaluating arthroscopic skills.

METHODS

The study collected simulator performance scores, consisting of "Triangulation Right Hand", "Triangulation Left Hand", "Catch the Stars" and "Three Rings" and DASS scores from 22 participants (11 novices, 11 experts). These scores were obtained while participants underwent a structured program of exercises for the fundamentals of arthroscopic surgery training (FAST) and knee module using a simulator-based arthroscopy device. During the evaluation, data on oxy-hemoglobin and deoxy-hemoglobin levels in the prefrontal cortex were collected using the Functional Near-Infrared Spectroscopy (fNIRS) imaging system. Performance scores, DASS scores, and fNIRS data were subsequently analyzed to determine any correlation between performance and cortex activity.

RESULTS

The simulator performance scores and the DASSPart2 scores were significantly higher in the expert group compared to the novice group (200.1 ± 28.5 vs 172.5 ± 48.9, p = 0.04 and 9.4 ± 5.6 vs. 5.4 ± 5.6 p = 0.02). In the expert group, fNIRS data showed a significantly lower prefrontal cortex activation during fundamental tasks in the FAST module, indicating significantly more efficient mental resource use.

CONCLUSION

The analysis of cognitive workload changes during simulation-based arthroscopy training revealed a significant correlation between the trainees' DASS scores and fNIRS data. This correlation suggests the potential use of fNIRS data and DASS scores as additional metrics to create adaptive training protocols for each participant. By incorporating these metrics, the training process can be optimized, leading to more efficient arthroscopic training and better preparedness for clinical operations.

LEVEL OF EVIDENCE

III.

Learning based motion artifacts processing in fNIRS: a mini review

This paper provides a concise review of learning-based motion artifacts (MA) processing methods in functional near-infrared spectroscopy (fNIRS), highlighting the challenges of maintaining optimal contact during subject movement, which can lead to MA and compromise data integrity. Traditional strategies often result in reduced reliability of the hemodynamic response and statistical power. Recognizing the limited number of studies focusing on learning-based MA removal, we examine 315 studies, identifying seven pertinent to our focus area. We discuss the current landscape of learning-based MA correction methods and highlight research gaps. Noting the absence of standard evaluation metrics for quality assessment of MA correction, we suggest a novel framework, integrating signal and model quality considerations and employing metrics like ΔSignal-to-Noise Ratio (ΔSNR), confusion matrix, and Mean Squared Error. This work aims to facilitate the application of learning-based methodologies to fNIRS and improve the accuracy and reliability of neurovascular studies.

Assessment of Surgical Tasks Using Neuroimaging Dataset (ASTaUND)

Functional near-infrared spectroscopy (fNIRS) is a neuroimaging tool for studying brain activity in mobile subjects. Open-access fNIRS datasets are limited to simple and/or motion-restricted tasks. Here, we report a fNIRS dataset acquired on mobile subjects performing Fundamentals of Laparoscopic Surgery (FLS) tasks in a laboratory environment. Demonstrating competency in the FLS tasks is a prerequisite for board certification in general surgery in the United States. The ASTaUND data set was acquired over four different studies. We provide the relevant information about the hardware, FLS task execution protocols, and subject demographics to facilitate the use of this open-access data set. We also provide the concurrent FLS scores, a quantitative metric for surgical skill assessment developed by the FLS committee. This data set is expected to support the growing field of assessing surgical skills via neuroimaging data and provide an example of data processing pipeline for use in realistic, non-restrictive environments.

Optical neuroimaging and neurostimulation in surgical training and assessment: A state-of-the-art review

Introduction

Functional near-infrared spectroscopy (fNIRS) is a non-invasive optical neuroimaging technique used to assess surgeons' brain function. The aim of this narrative review is to outline the effect of expertise, stress, surgical technology, and neurostimulation on surgeons' neural activation patterns, and highlight key progress areas required in surgical neuroergonomics to modulate training and performance.

Methods

A literature search of PubMed and Embase was conducted to identify neuroimaging studies using fNIRS and neurostimulation in surgeons performing simulated tasks.

Results

Novice surgeons exhibit greater haemodynamic responses across the pre-frontal cortex than experts during simple surgical tasks, whilst expert surgical performance is characterized by relative prefrontal attenuation and upregulation of activation foci across other regions such as the supplementary motor area. The association between PFC activation and mental workload follows an inverted-U shaped curve, activation increasing then attenuating past a critical inflection point at which demands outstrip cognitive capacity Neuroimages are sensitive to the impact of laparoscopic and robotic tools on cognitive workload, helping inform the development of training programs which target neural learning curves. FNIRS differs in comparison to current tools to assess proficiency by depicting a cognitive state during surgery, enabling the development of cognitive benchmarks of expertise. Finally, neurostimulation using transcranial direct-current-stimulation may accelerate skill acquisition and enhance technical performance.

Conclusion

FNIRS can inform the development of surgical training programs which modulate stress responses, cognitive learning curves, and motor skill performance. Improved data processing with machine learning offers the possibility of live feedback regarding surgeons' cognitive states during operative procedures.

Use of neuroimaging to measure neurocognitive engagement in health professions education: a scoping review

PURPOSE

To map the current literature on functional neuroimaging use in medical education research as a novel measurement modality for neurocognitive engagement, learning, and expertise development.

METHOD

We searched PubMed, Embase, Cochrane, ERIC, and Web of Science, and hand-searched reference lists of relevant articles on April 4, 2019, and updated the search on July 7, 2020. Two authors screened the abstracts and then full-text articles for eligibility based on inclusion criteria. The data were then charted, synthesized, and analyzed descriptively.

RESULTS

Sixty-seven articles published between 2007 and 2020 were included in this scoping review. These studies used three main neuroimaging modalities: functional magnetic resonance imaging, functional near-infrared spectroscopy, and electroencephalography. Most of the publications (90%, n = 60) were from the last 10 years (2011-2020). Although these studies were conducted in 16 countries, 68.7% (n = 46) were from three countries: the USA (n = 21), UK (n = 15), and Canada (n = 10). These studies were mainly non-experimental (74.6%, n = 50). Most used neuroimaging techniques to examine psychomotor skill development (57%, n = 38), but several investigated neurocognitive correlates of clinical reasoning skills (22%, n = 15).

CONCLUSION

This scoping review maps the available literature on functional neuroimaging use in medical education. Despite the heterogeneity in research questions, study designs, and outcome measures, we identified a few common themes. Included studies are encouraging of the potential for neuroimaging to complement commonly used measures in education research and may help validate/challenge established theoretical assumptions and provide insight into training methods. This review highlighted several areas for further research. The use of these emerging technologies appears ripe for developing precision education, establishing viable study protocols for realistic operational settings, examining team dynamics, and exploring applications for real-time monitoring/intervention during critical clinical tasks.

Deep learning-based motion artifact removal in functional near-infrared spectroscopy

Significance: Functional near-infrared spectroscopy (fNIRS), a well-established neuroimaging technique, enables monitoring cortical activation while subjects are unconstrained. However, motion artifact is a common type of noise that can hamper the interpretation of fNIRS data. Current methods that have been proposed to mitigate motion artifacts in fNIRS data are still dependent on expert-based knowledge and the post hoc tuning of parameters. Aim: Here, we report a deep learning method that aims at motion artifact removal from fNIRS data while being assumption free. To the best of our knowledge, this is the first investigation to report on the use of a denoising autoencoder (DAE) architecture for motion artifact removal. Approach: To facilitate the training of this deep learning architecture, we (i) designed a specific loss function and (ii) generated data to mimic the properties of recorded fNIRS sequences. Results: The DAE model outperformed conventional methods in lowering residual motion artifacts, decreasing mean squared error, and increasing computational efficiency. Conclusion: Overall, this work demonstrates the potential of deep learning models for accurate and fast motion artifact removal in fNIRS data.

Studying Brain Activation during Skill Acquisition via Robot-Assisted Surgery Training

Robot-assisted surgery systems are a recent breakthrough in minimally invasive surgeries, offering numerous benefits to both patients and surgeons including, but not limited to, greater visualization of the operation site, greater precision during operation and shorter hospitalization times. Training on robot-assisted surgery (RAS) systems begins with the use of high-fidelity simulators. Hence, the increasing demand of employing RAS systems has led to a rise in using RAS simulators to train medical doctors. The aim of this study was to investigate the brain activity changes elicited during the skill acquisition of resident surgeons by measuring hemodynamic changes from the prefrontal cortex area via a neuroimaging sensor, namely, functional near-infrared spectroscopy (fNIRS). Twenty-four participants, who are resident medical doctors affiliated with different surgery departments, underwent an RAS simulator training during this study and completed the sponge suturing tasks at three different difficulty levels in two consecutive sessions/blocks. The results reveal that cortical oxygenation changes in the prefrontal cortex were significantly lower during the second training session (Block 2) compared to the initial training session (Block 1) (p < 0.05).

Functional Brain Imaging Reliably Predicts Bimanual Motor Skill Performance in a Standardized Surgical Task

Currently, there is a dearth of objective metrics for assessing bi-manual motor skills, which are critical for high-stakes professions such as surgery. Recently, functional near-infrared spectroscopy (fNIRS) has been shown to be effective at classifying motor task types, which can be potentially used for assessing motor performance level. In this work, we use fNIRS data for predicting the performance scores in a standardized bi-manual motor task used in surgical certification and propose a deep-learning framework 'Brain-NET' to extract features from the fNIRS data. Our results demonstrate that the Brain-NET is able to predict bi-manual surgical motor skills based on neuroimaging data accurately ( R2=0.73). Furthermore, the classification ability of the Brain-NET model is demonstrated based on receiver operating characteristic (ROC) curves and area under the curve (AUC) values of 0.91. Hence, these results establish that fNIRS associated with deep learning analysis is a promising method for a bedside, quick and cost-effective assessment of bi-manual skill levels.

How Wearable Technology Can Facilitate AI Analysis of Surgical Videos

Operative video has great potential to enable instant replays of critical surgical decisions for training and quality review. Recently, artificial intelligence (AI) has shown early promise as a method of enabling efficient video review, analysis, and segmentation. Despite the progress with AI analysis of surgical videos, more work needs to be done to improve the accuracy and efficiency of AI-driven video analysis. At a recent consensus conference held on July 10-11, 2020, 8 research teams shared their work using AI for surgical video analysis. Four of the teams showcased the utility of wearable technology in providing objective surgical metrics. Data from these technologies were shown to pinpoint important cognitive and motor actions during operative tasks and procedures. The results support the utility of wearable technology to facilitate efficient and accurate video analysis and segmentation.

Use of prefrontal cortex activity as a measure of learning curve in surgical novices: results of a single blind randomised controlled trial

BACKGROUND

Neurobiological feedback in surgical training could translate to better educational outcomes such as measures of learning curve. This work examined the variation in brain activation of medical students when performing laparoscopic tasks before and after a training workshop, using functional near-infrared spectroscopy (fNIRS).

METHODS AND PROCEDURES

This single blind randomised controlled trial examined the prefrontal cortex activity (PFCA) differences in two groups of novice medical students during the acquisition of four laparoscopic tasks. Both groups were shown a basic tutorial video, with the "Trained-group" receiving an additional standardised one-to-one training on the tasks. The PFCA was measured pre- and post-intervention using a portable fNIRS device and reported as mean total oxygenated hemoglobin (HbOµm). Primary outcome of the study is the difference in HbOµm between post- and pre-intervention readings for each of the four laparoscopic tasks. The pre- and post-intervention laparoscopic tasks were recorded and assessed by two blinded individual assessors for objective scores of the performance.

RESULTS

16 Trained and 16 Untrained, right-handed medical students with an equal sex distribution and comparable age distribution were recruited. Trained group had an attenuated left PFCA in the "Precision cutting" (p = 0.007) task compared to the Untrained group. Subgroup analysis by sex revealed attenuation in left PFCA in Trained females compared to Untrained females across two laparoscopic tasks: "Peg transfer" (p = 0.005) and "Precision cutting" (p = 0.003). No significant PFCA attenuation was found in male students who underwent training compared to Untrained males.

CONCLUSION

A standardised laparoscopic training workshop promoted greater PFCA attenuation in female medical students compared to males. This suggests that female and male students respond differently to the same instructional approach. Implications include a greater focus on one-to-one surgical training for female students and use of PFCA attenuation as a form of neurobiological feedback in surgical training.

A machine learning approach to predict surgical learning curves

BACKGROUND

Contemporary surgical training programs rely on the repetition of selected surgical motor tasks. Such methodology is inherently open ended with no control on the time taken to attain a set level of proficiency, given the trainees' intrinsic differences in initial skill levels and learning abilities. Hence, an efficient training program should aim at tailoring the surgical training protocols to each trainee. In this regard, a predictive model using information from the initial learning stage to predict learning curve characteristics should facilitate the whole surgical training process.

METHODS

This paper analyzes learning curve data to train a multivariate supervised machine learning model. One factor is extracted to define the trainees' learning ability. An unsupervised machine learning model is also utilized for trainee classification. When established, the model can predict robustly the learning curve characteristics based on the first few trials.

RESULTS

We show that the information present in the first 10 trials of surgical tasks can be utilized to predict the number of trials required to achieve proficiency (R²=0.72) and the final performance level (R²=0.89). Furthermore, only a single factor, learning index, is required to describe the learning process and to classify learners with unique learning characteristics.

CONCLUSION

Using machine learning models, we show, for the first time, that the first few trials contain sufficient information to predict learning curve characteristics and that a single factor can capture the complex learning behavior. Using such models holds the potential for personalization of training regimens, leading to greater efficiency and lower costs.