Assessing bimanual motor skills with optical neuroimaging

Using neuroimaging to assess brain activity and areas associated with surgical skills: a systematic review

BACKGROUND

Surgical skills acquisition is under continuous development due to the emergence of new technologies, and there is a need for assessment tools to develop along with these. A range of neuroimaging modalities has been used to map the functional activation of brain networks while surgeons acquire novel surgical skills. These have been proposed as a method to provide a deeper understanding of surgical expertise and offer new possibilities for the personalized training of future surgeons. With studies differing in modalities, outcomes, and surgical skills there is a need for a systematic review of the evidence. This systematic review aims to summarize the current knowledge on the topic and evaluate the potential use of neuroimaging in surgical education.

METHODS

We conducted a systematic review of neuroimaging studies that mapped functional brain activation while surgeons with different levels of expertise learned and performed technical and non-technical surgical tasks. We included all studies published before July 1st, 2023, in MEDLINE, EMBASE and WEB OF SCIENCE.

RESULTS

38 task-based brain mapping studies were identified, consisting of randomized controlled trials, case-control studies, and observational cohort or cross-sectional studies. The studies employed a wide range of brain mapping modalities, including electroencephalography, functional magnetic resonance imaging, positron emission tomography, and functional near-infrared spectroscopy, activating brain areas involved in the execution and sensorimotor or cognitive control of surgical skills, especially the prefrontal cortex, supplementary motor area, and primary motor area, showing significant changes between novices and experts.

CONCLUSION

Functional neuroimaging can reveal how task-related brain activity reflects technical and non-technical surgical skills. The existing body of work highlights the potential of neuroimaging to link task-related brain activity patterns with the individual level of competency or improvement in performance after training surgical skills. More research is needed to establish its validity and usefulness as an assessment tool.

Effects of age and flight experience on prefrontal cortex activity in airline pilots: An fNIRS study

It is essential for airlines to have a deep understanding of the cognitive impact of aging among pilots. The current literature on executive function indicates that compensatory mechanisms in the brain may counteract age-related cognitive decline, at least up to certain task load levels. However, few studies have been administered to evaluate changes in aircrew competence as they age. The present study focuses on dorsolateral prefrontal cortex (DLPFC) activity as it is implicated in cognitive performance and working memory, which are associated with skill proficiency. We measured the DLPFC activity for airline pilots, including trainees, during maneuvering using a flight simulator. Our preliminary results indicated that only expert (aged) pilots demonstrated higher activity of the left DLPFC than the right one. However, for youth trainees, not only was the error rate high while using the flight simulator, but the activity of the DLFPC was also lower than that of the expert pilots, and there was no statistically significant difference between the left and right DLPFC. Although these findings partially differ from those reported in previous studies on age-related changes, it is evident that training as an airline pilot for over 20 years may affect such results. We believe that this noninvasive approach to objective quantification of skill will facilitate the development of effective assessment competence in aging.

Validation of a Virtual Simulator With Haptic Feedback for Endotracheal Intubation Training

INTRODUCTION

Endotracheal intubation (ETI) is a procedure that varies in difficulty because of patient characteristics and clinical conditions. Existing physical simulators do not encompass these variations. The Virtual Airway Skills Trainer for Endotracheal Intubation (VAST-ETI) was developed to provide different patient characteristics and high-fidelity haptic feedback to improve training.

METHODS

We demonstrate the effectiveness of VAST-ETI as a training and evaluation tool for ETI. Construct validation was evaluated by scoring the performance of experts ( N = 15) and novices ( N = 15) on the simulator to ensure its ability to distinguish technical proficiency. Convergent and predictive validity were evaluated by performing a learning curve study, in which a group of novices ( N = 7) were trained for 2 weeks using VAST-ETI and then compared with a control group ( N = 9).

RESULTS

The VAST-ETI was able to distinguish between expert and novice based on mean simulator scores ( t [88] = -6.61, P < 0.0005). When used during repeated practice, individuals demonstrated a significant increase in their score on VAST-ETI over the learning period ( F [11,220] = 7206, P < 0.001); however when compared with a control group, there was not a significant interaction effect on the simulator score. There was a significant difference between the simulator-trained and control groups ( t [12.85] = -2.258, P = 0.042) when tested in the operating room.

CONCLUSIONS

Our results demonstrate the effectiveness of virtual simulation with haptic feedback for assessing performance and training of ETI. The simulator was not able to differentiate performance between more experienced trainees and experts because of limits in simulator difficulty.

Development of performance and learning rate evaluation models in robot-assisted surgery using electroencephalography and eye-tracking

The existing performance evaluation methods in robot-assisted surgery (RAS) are mainly subjective, costly, and affected by shortcomings such as the inconsistency of results and dependency on the raters' opinions. The aim of this study was to develop models for an objective evaluation of performance and rate of learning RAS skills while practicing surgical simulator tasks. The electroencephalogram (EEG) and eye-tracking data were recorded from 26 subjects while performing Tubes, Suture Sponge, and Dots and Needles tasks. Performance scores were generated by the simulator program. The functional brain networks were extracted using EEG data and coherence analysis. Then these networks, along with community detection analysis, facilitated the extraction of average search information and average temporal flexibility features at 21 Brodmann areas (BA) and four band frequencies. Twelve eye-tracking features were extracted and used to develop linear random intercept models for performance evaluation and multivariate linear regression models for the evaluation of the learning rate. Results showed that subject-wise standardization of features improved the R2 of the models. Average pupil diameter and rate of saccade were associated with performance in the Tubes task (multivariate analysis; p-value = 0.01 and p-value = 0.04, respectively). Entropy of pupil diameter was associated with performance in Dots and Needles task (multivariate analysis; p-value = 0.01). Average temporal flexibility and search information in several BAs and band frequencies were associated with performance and rate of learning. The models may be used to objectify performance and learning rate evaluation in RAS once validated with a broader sample size and tasks.

Neural correlates of fine motor grasping skills: Longitudinal insights into motor cortex activation using fNIRS

BACKGROUND

Motor learning is essential for performing specific tasks and progresses through distinct stages, including the rapid learning phase (initial skill acquisition), the consolidation phase (skill refinement), and the stable performance phase (skill mastery and maintenance). Understanding the cortical activation dynamics during these stages can guide targeted rehabilitation interventions.

METHODS

In this longitudinal randomized controlled trial, functional near-infrared spectroscopy was used to explore the temporal dynamics of cortical activation in hand-related motor learning. Thirty-one healthy right-handed individuals were randomly assigned to perform either easy or intricate motor tasks with their non-dominant hand over 10 days. We conducted 10 monitoring sessions to track cortical activation in the right hemisphere (according to lateralization principles, the primary hemisphere for motor control) and evaluated motor proficiency concurrently.

RESULTS

The study delineated three stages of nondominant hand motor learning: rapid learning (days 1 and 2), consolidation (days 3-7), and stable performance (days 8-10). There was a power-law enhancement of motor skills correlated with learning progression. Sustained activation was observed in the supplementary motor area (SMA) and parietal lobe (PL), whereas activation in the right primary motor cortex (M1R) and dorsolateral prefrontal cortex (PFCR) decreased. These cortical activation patterns exhibited a high correlation with the augmentation of motor proficiency.

CONCLUSIONS

The findings suggest that early rehabilitation interventions, such as transcranial magnetic stimulation and transcranial direct current stimulation (tDCS), could be optimally directed at M1 and PFC in the initial stages. In contrast, SMA and PL can be targeted throughout the motor learning process. This research illuminates the path for developing tailored motor rehabilitation interventions based on specific stages of motor learning.

NEW AND NOTEWORTHY

In an innovative approach, our study uniquely combines a longitudinal design with the robustness of generalized estimating equations (GEEs). With the synergy of functional near-infrared spectroscopy (fNIRS) and the Minnesota Manual Dexterity Test (MMDT) paradigm, we precisely trace the evolution of neural resources during complex, real-world fine-motor task learning. Centering on right-handed participants using their nondominant hand magnifies the intricacies of right hemisphere spatial motor processing. We unravel the brain's dynamic response throughout motor learning stages and its potent link to motor skill enhancement. Significantly, our data point toward the early-phase rehabilitation potential of TMS and transcranial direct current stimulation on the M1 and PFC regions. Concurrently, SMA and PL appear poised to benefit from ongoing interventions during the entire learning curve. Our findings carve a path for refined motor rehabilitation strategies, underscoring the importance of timely noninvasive brain stimulation treatments.

Learning based motion artifacts processing in fNIRS: a mini review

This paper provides a concise review of learning-based motion artifacts (MA) processing methods in functional near-infrared spectroscopy (fNIRS), highlighting the challenges of maintaining optimal contact during subject movement, which can lead to MA and compromise data integrity. Traditional strategies often result in reduced reliability of the hemodynamic response and statistical power. Recognizing the limited number of studies focusing on learning-based MA removal, we examine 315 studies, identifying seven pertinent to our focus area. We discuss the current landscape of learning-based MA correction methods and highlight research gaps. Noting the absence of standard evaluation metrics for quality assessment of MA correction, we suggest a novel framework, integrating signal and model quality considerations and employing metrics like ΔSignal-to-Noise Ratio (ΔSNR), confusion matrix, and Mean Squared Error. This work aims to facilitate the application of learning-based methodologies to fNIRS and improve the accuracy and reliability of neurovascular studies.

Assessment of Surgical Tasks Using Neuroimaging Dataset (ASTaUND)

Functional near-infrared spectroscopy (fNIRS) is a neuroimaging tool for studying brain activity in mobile subjects. Open-access fNIRS datasets are limited to simple and/or motion-restricted tasks. Here, we report a fNIRS dataset acquired on mobile subjects performing Fundamentals of Laparoscopic Surgery (FLS) tasks in a laboratory environment. Demonstrating competency in the FLS tasks is a prerequisite for board certification in general surgery in the United States. The ASTaUND data set was acquired over four different studies. We provide the relevant information about the hardware, FLS task execution protocols, and subject demographics to facilitate the use of this open-access data set. We also provide the concurrent FLS scores, a quantitative metric for surgical skill assessment developed by the FLS committee. This data set is expected to support the growing field of assessing surgical skills via neuroimaging data and provide an example of data processing pipeline for use in realistic, non-restrictive environments.

Video game experience affects performance, cognitive load, and brain activity in laparoscopic surgery training

Objectives

Video games can be a valuable tool for surgery training. Individuals who interact or play video games tend to have a better visuospatial ability when compared to non-gamers. Numerous studies suggest that video game experience is associated with faster acquisition, greater sharpening, and longer retention of laparoscopic skills. Given the neurocognitive complexity of surgery skill, multimodal approaches are required to understand how video game playing enhances laparoscopy skill.

Material and Methods

Twenty-seven students with no laparoscopy experience and varying levels of video game experience performed standard laparoscopic training tasks. Their performance, subjective cognitive loading, and prefrontal cortical activity were recorded and analyzed. As a reference point to use in comparing the two novice groups, we also included data from 13 surgeons with varying levels of laparoscopy experience and no video game experience.

Results

Results indicated that video game experience was correlated with higher performance (R2 = 0.22, p <0.01) and lower cognitive load (R2 = 0.21, p <0.001), and the prefrontal cortical activation of students with gaming experience was relatively lower than those without gaming experience. In terms of these variables, gaming experience in novices tended to produce effects similar to those of laparoscopy experience in surgeons.

Conclusion

Our results suggest that along the dimensions of performance, cognitive load, and brain activity, the effects of video gaming experience on novice laparoscopy trainees are similar to those of real-world laparoscopy experience on surgeons. We believe that the neural underpinnings of surgery skill and its links with gaming experience need to be investigated further using wearable functional brain imaging.

Optical neuroimaging and neurostimulation in surgical training and assessment: A state-of-the-art review

Introduction

Functional near-infrared spectroscopy (fNIRS) is a non-invasive optical neuroimaging technique used to assess surgeons' brain function. The aim of this narrative review is to outline the effect of expertise, stress, surgical technology, and neurostimulation on surgeons' neural activation patterns, and highlight key progress areas required in surgical neuroergonomics to modulate training and performance.

Methods

A literature search of PubMed and Embase was conducted to identify neuroimaging studies using fNIRS and neurostimulation in surgeons performing simulated tasks.

Results

Novice surgeons exhibit greater haemodynamic responses across the pre-frontal cortex than experts during simple surgical tasks, whilst expert surgical performance is characterized by relative prefrontal attenuation and upregulation of activation foci across other regions such as the supplementary motor area. The association between PFC activation and mental workload follows an inverted-U shaped curve, activation increasing then attenuating past a critical inflection point at which demands outstrip cognitive capacity Neuroimages are sensitive to the impact of laparoscopic and robotic tools on cognitive workload, helping inform the development of training programs which target neural learning curves. FNIRS differs in comparison to current tools to assess proficiency by depicting a cognitive state during surgery, enabling the development of cognitive benchmarks of expertise. Finally, neurostimulation using transcranial direct-current-stimulation may accelerate skill acquisition and enhance technical performance.

Conclusion

FNIRS can inform the development of surgical training programs which modulate stress responses, cognitive learning curves, and motor skill performance. Improved data processing with machine learning offers the possibility of live feedback regarding surgeons' cognitive states during operative procedures.

Suppression of neuroinflammation and α-synuclein oligomerization by rotarod walking exercise in subacute MPTP model of Parkinson's disease

Parkinson's disease (PD) belongs to an α-synucleinopathy and manifests motor dysfunction attributed to nigrostriatal dopaminergic degeneration. In clinical practice, the beneficial role of physical therapy such as motor skill learning training has been recognized in PD-linked motor defects. Nevertheless, the disease-modifying effects of motor skill learning training on PD-related pathology remain unclear. Here, we investigated the disease-modifying effects of rotarod walking exercise (RWE), a modality of motor skill learning training, in a subacute 1-methyl-4-phenyl-1,2,3,6-tetrahydropyridine (MPTP) mouse model of PD. In motor function and dopaminergic degeneration, RWE improved MPTP-induced deficits. In addition, RWE enhanced the expression of neurotrophic factors BDNF/GDNF, PGC1-α, Nurr1, and p-AMPK, thereby recovering dopaminergic neuronal cell death. Moreover, RWE inhibited microglial activation and the expression of pro-inflammatory markers, such as p-IκBα, iNOS, IL-1β, TNF-α, and cathepsin D, while elevating anti-inflammatory IL-10 and TGF-β. RWE also decreased oxidative stress markers in the substantia nigra, such as 4-HNE and 8-OHdG-positive cells, while increasing Nrf2-controlled antioxidant enzymes. Regarding the effect of RWE on α-synuclein, it reduced the monomer/oligomer forms of α-synuclein and phosphorylation at serine 129. Further mechanistic studies revealed that RWE suppressed the expression of matrix metalloproteinase-3 and p-GSK3β (Y216), which play key roles in α-synuclein aggregation. These data collectively suggest that inhibition of neuroinflammation and α-synuclein oligomerization by RWE may contribute to the improvement of PD pathology.

Deep learning-based motion artifact removal in functional near-infrared spectroscopy

Significance: Functional near-infrared spectroscopy (fNIRS), a well-established neuroimaging technique, enables monitoring cortical activation while subjects are unconstrained. However, motion artifact is a common type of noise that can hamper the interpretation of fNIRS data. Current methods that have been proposed to mitigate motion artifacts in fNIRS data are still dependent on expert-based knowledge and the post hoc tuning of parameters. Aim: Here, we report a deep learning method that aims at motion artifact removal from fNIRS data while being assumption free. To the best of our knowledge, this is the first investigation to report on the use of a denoising autoencoder (DAE) architecture for motion artifact removal. Approach: To facilitate the training of this deep learning architecture, we (i) designed a specific loss function and (ii) generated data to mimic the properties of recorded fNIRS sequences. Results: The DAE model outperformed conventional methods in lowering residual motion artifacts, decreasing mean squared error, and increasing computational efficiency. Conclusion: Overall, this work demonstrates the potential of deep learning models for accurate and fast motion artifact removal in fNIRS data.

Deep learning in fNIRS: a review

Significance: Optical neuroimaging has become a well-established clinical and research tool to monitor cortical activations in the human brain. It is notable that outcomes of functional near-infrared spectroscopy (fNIRS) studies depend heavily on the data processing pipeline and classification model employed. Recently, deep learning (DL) methodologies have demonstrated fast and accurate performances in data processing and classification tasks across many biomedical fields. Aim: We aim to review the emerging DL applications in fNIRS studies. Approach: We first introduce some of the commonly used DL techniques. Then, the review summarizes current DL work in some of the most active areas of this field, including brain-computer interface, neuro-impairment diagnosis, and neuroscience discovery. Results: Of the 63 papers considered in this review, 32 report a comparative study of DL techniques to traditional machine learning techniques where 26 have been shown outperforming the latter in terms of the classification accuracy. In addition, eight studies also utilize DL to reduce the amount of preprocessing typically done with fNIRS data or increase the amount of data via data augmentation. Conclusions: The application of DL techniques to fNIRS studies has shown to mitigate many of the hurdles present in fNIRS studies such as lengthy data preprocessing or small sample sizes while achieving comparable or improved classification accuracy.

Directed information flow during laparoscopic surgical skill acquisition dissociated skill level and medical simulation technology

Virtual reality (VR) simulator has emerged as a laparoscopic surgical skill training tool that needs validation using brain-behavior analysis. Therefore, brain network and skilled behavior relationship were evaluated using functional near-infrared spectroscopy (fNIRS) from seven experienced right-handed surgeons and six right-handed medical students during the performance of Fundamentals of Laparoscopic Surgery (FLS) pattern of cutting tasks in a physical and a VR simulator. Multiple regression and path analysis (MRPA) found that the FLS performance score was statistically significantly related to the interregional directed functional connectivity from the right prefrontal cortex to the supplementary motor area with F (2, 114) = 9, p < 0.001, and R² = 0.136. Additionally, a two-way multivariate analysis of variance (MANOVA) found a statistically significant effect of the simulator technology on the interregional directed functional connectivity from the right prefrontal cortex to the left primary motor cortex (F (1, 15) = 6.002, p = 0.027; partial η² = 0.286) that can be related to differential right-lateralized executive control of attention. Then, MRPA found that the coefficient of variation (CoV) of the FLS performance score was statistically significantly associated with the CoV of the interregionally directed functional connectivity from the right primary motor cortex to the left primary motor cortex and the left primary motor cortex to the left prefrontal cortex with F (2, 22) = 3.912, p = 0.035, and R² = 0.262. This highlighted the importance of the efference copy information from the motor cortices to the prefrontal cortex for postulated left-lateralized perceptual decision-making to reduce behavioral variability.

Neuroenhancement of surgeons during robotic suturing

BACKGROUND

The initial phases of robotic surgical skills acquisition are associated with poor technical performance, such as low knot-tensile strength (KTS). Transcranial direct-current stimulation (tDCS) can improve force and accuracy in motor tasks but research in surgery is limited to open and laparoscopic tasks in students. More recently, robotic surgery has gained traction and is now the most common approach for certain procedures (e.g. prostatectomy). Early-phase robotic suturing performance is dependent on prefrontal cortex (PFC) activation, and this study aimed to determine whether performance can be improved with prefrontal tDCS.

METHODS

Fifteen surgical residents were randomized to either active then sham tDCS or sham then active tDCS, in two counterbalanced sessions in a double-blind crossover study. Within each session, participants performed a robotic suturing task repeated in three blocks: pre-, intra- and post-tDCS. During the intra-tDCS block, participants were randomized to either active tDCS (2 mA for 15 min) to the PFC or sham tDCS. Primary outcome measures of technical quality included KTS and error scores.

RESULTS

Significantly faster completion times were observed longitudinally, regardless of active (p < 0.001) or sham stimulation (p < 0.001). KTS was greater following active compared to sham stimulation (median: active = 44.35 N vs. sham = 27.12 N, p < 0.001). A significant reduction in error scores from "pre-" to "post-" (p = 0.029) were only observed in the active group.

CONCLUSION

tDCS could reduce error and enhance KTS during robotic suturing and warrants further exploration as an adjunct to robotic surgical training.

Neuroergonomic assessment of developmental coordination disorder

Until recently, neural assessments of gross motor coordination could not reliably handle active tasks, particularly in realistic environments, and offered a narrow understanding of motor-cognition. By applying a comprehensive neuroergonomic approach using optical mobile neuroimaging, we probed the neural correlates of motor functioning in young people with Developmental Coordination Disorder (DCD), a motor-learning deficit affecting 5-6% of children with lifelong complications. Neural recordings using fNIRS were collected during active ambulatory behavioral task execution from 37 Typically Developed and 48 DCD Children who performed cognitive and physical tasks in both single and dual conditions. This is the first of its kind study targeting regions of prefrontal cortical dysfunction for identification of neuropathophysiology for DCD during realistic motor tasks and is one of the largest neuroimaging study (across all modalities) involving DCD. We demonstrated that DCD is a motor-cognitive disability, as gross motor /complex tasks revealed neuro-hemodynamic deficits and dysfunction within the right middle and superior frontal gyri of the prefrontal cortex through functional near infrared spectroscopy. Furthermore, by incorporating behavioral performance, decreased neural efficiency in these regions were revealed in children with DCD, specifically during motor tasks. Lastly, we provide a framework, evaluating disorder impact in ecologically valid contexts to identify when and for whom interventional approaches are most needed and open the door for precision therapies.

Neuromonitoring Correlates of Expertise Level in Surgical Performers: A Systematic Review

Surgical expertise does not have a clear definition and is often culturally associated with power, authority, prestige, and case number rather than more objective proxies of excellence. Multiple models of expertise progression have been proposed including the Dreyfus model, however, they all currently require subjective evaluation of skill. Recently, efforts have been made to improve the ways in which surgical excellence is measured and expertise is defined using artificial intelligence, video recordings, and accelerometers. However, these aforementioned methods of assessment are still subjective or indirect proxies of expertise, thus uncovering the neural mechanisms that differentiate expert surgeons from trainees may enhance the objectivity of surgical expertise validation. In fact, some researchers have already suggested that their neural imaging-based expertise classification methods outperform currently used methods of surgical skill certification such as the Fundamentals of Laparoscopic Surgery (FLS) scores. Such imaging biomarkers would not only help better identify the highest performing surgeons, but could also improve residency programs by providing more objective, evidence-based feedback and developmental milestones for those in training and perhaps act as a marker of surgical potential in medical students. Despite the potential advantages of using neural imaging in the assessment of surgical expertise, this field of research remains in its infancy. This systematic review identifies studies that have applied neuromonitoring in assessing surgical skill across levels of expertise. The goals of this review are to identify (1) the strongest neural indicators of surgical expertise, (2) the limitations of the current literature on this subject, (3) the most sensible future directions for further study. We found substantial evidence that surgical expertise can be delineated by differential activation and connectivity in the prefrontal cortex (PFC) across multiple task and neuroimaging modalities. Specifically, novices tend to have greater PFC activation than experts under standard conditions in bimanual and decision-making tasks. However, under high temporal demand tasks, experts had increased PFC activation whereas novices had decreased PFC activation. Common limitations uncovered in this review were that task difficulty was often insufficient to delineate between residents and attending. Moreover, attending level involvement was also low in multiple studies which may also have contributed to this issue. Most studies did not analyze the ability of their neuromonitoring findings to accurately classify subjects by level of expertise. Finally, the predominance of fNIRS as the neuromonitoring modality limits our ability to uncover the neural correlates of surgical expertise in non-cortical brain regions. Future studies should first strive to address these limitations. In the longer term, longitudinal within-subjects design over the course of a residency or even a career will also advance the field. Although logistically arduous, such studies would likely be most beneficial in demonstrating effects of increasing surgical expertise on regional brain activation and inter-region connectivity.

Changes in brain activity of trainees during laparoscopic surgical virtual training assessed with electroencephalography

OBJECTIVE

Evaluate changes in brain activity of trainees during laparoscopic surgical training from electroencephalographic (EEG) signals in an ecological scenario with few restrictions for the user. Design Longitudinal study with two follow-up measurements in the first and last session of a 4-week training with LapSim laparoscopic surgery simulator. Variables analyzed include EEG neuronal activations in theta and alpha bands, tasks performance measures, and subjective measures such as perception of mental workload. Setting Medical School, Universidad de Antioquia, Medellin, Colombia. Participants First-year surgical residents (n = 16, age = 28.0 ± 2.6 years old, right-handed, 9 females) RESULTS: Significant improvements in tasks performance were found together with changes in neuronal activity over frontal and parietal cortex. These changes were also correlated with task performance through training sessions.

CONCLUSIONS

The use of neurophysiological measures such as electroencephalography combined with source separation techniques allows evaluating neural changes associated with motor training. The experiment proposed in this work establishes less controlled recording conditions leading to a more realistic analysis scenario to cognitive assessment in residents training.

Neuroimaging guided tES to facilitate complex laparoscopic surgical tasks - insights from functional near-infrared spectroscopy

Fundamentals of Laparoscopic Surgery (FLS) is a prerequisite for board certification in general surgery in the USA. In FLS, the suturing task with intracorporeal knot tying is considered the most complex task. Transcranial direct current stimulation (tDCS) of the dorsolateral prefrontal cortex (PFC) has been shown to facilitate FLS surgical skill acquisition where 2mA tDCS for 15min with the anode over F3 (10/10 EEG montage) and cathode over F4 has improved performance score in an open knot-tying task. Since PFC has a functional organization related to the hierarchy of cognitive control, we performed functional near-infrared spectroscopy (fNIRS) to investigate PFC sub-domain activation during a more complex FLS suturing task with intracorporeal knot tying. We performed fNIRS-based analysis using AtlasViewer software on two expert surgeons and four novice medical students. We found an average cortical activation mainly at the left frontopolar PFC across the experts, while the average cortical activation across the novices was primarily at the left pars opercularis of the inferior frontal gyrus and ventral premotor cortex, inferior parietal lobule, and supramarginal gyrus. Here, the average cortical activation across the novices included not only the cognitive control related brain regions but also motor control complexity related brain regions. Therefore, we present a computational pipeline to identify a 4x1 high-definition (HD) tDCS montage of motor complexity related PFC sub-regions using ROAST software.Clinical Relevance-A computational pipeline for fNIRS-guided tES to individualize electrode montage that may facilitate FLS surgical training in our future studies.

Neuroimaging Guided Transcranial Electrical Stimulation in Enhancing Surgical Skill Acquisition. Comment on Hung et al. The Efficacy of Transcranial Direct Current Stimulation in Enhancing Surgical Skill Acquisition: A Preliminary Meta-Analysis of Randomized Controlled Trials. Brain Sci. 2021, 11, 707

Surgical skill acquisition may be facilitated with a safe application of transcranial direct current stimulation (tDCS). A preliminary meta-analysis of randomized control trials showed that tDCS was associated with significantly better improvement in surgical performance than the sham control; however, meta-analysis does not address the mechanistic understanding. It is known from skill learning studies that the hierarchy of cognitive control shows a rostrocaudal axis in the frontal lobe where a shift from posterior to anterior is postulated to mediate progressively abstract, higher-order control. Therefore, optimizing the transcranial electrical stimulation to target surgical task-related brain activation at different stages of motor learning may provide the causal link to the learning behavior. This comment paper presents the computational approach for neuroimaging guided tDCS based on open-source software pipelines and an open-data of functional near-infrared spectroscopy (fNIRS) for complex motor tasks. We performed an fNIRS-based cortical activation analysis using AtlasViewer software that was used as the target for tDCS of the motor complexity-related brain regions using ROAST software. For future studies on surgical skill training, it is postulated that the higher complexity laparoscopic suturing with intracorporeal knot tying task may result in more robust activation of the motor complexity-related brain areas when compared to the lower complexity laparoscopic tasks.

Performance assessment using sensor technology

Over the past 30 years, there have been numerous, noteworthy successes in the development, validation, and implementation of clinical skills assessments. Despite this progress, the medical profession has barely scratched the surface towards developing assessments that capture the true complexity of hands-on skills in procedural medicine. This paper highlights the development implementation and new discoveries in performance metrics when using sensor technology to assess cognitive and technical aspects of hands-on skills.

Studying Brain Activation during Skill Acquisition via Robot-Assisted Surgery Training

Robot-assisted surgery systems are a recent breakthrough in minimally invasive surgeries, offering numerous benefits to both patients and surgeons including, but not limited to, greater visualization of the operation site, greater precision during operation and shorter hospitalization times. Training on robot-assisted surgery (RAS) systems begins with the use of high-fidelity simulators. Hence, the increasing demand of employing RAS systems has led to a rise in using RAS simulators to train medical doctors. The aim of this study was to investigate the brain activity changes elicited during the skill acquisition of resident surgeons by measuring hemodynamic changes from the prefrontal cortex area via a neuroimaging sensor, namely, functional near-infrared spectroscopy (fNIRS). Twenty-four participants, who are resident medical doctors affiliated with different surgery departments, underwent an RAS simulator training during this study and completed the sponge suturing tasks at three different difficulty levels in two consecutive sessions/blocks. The results reveal that cortical oxygenation changes in the prefrontal cortex were significantly lower during the second training session (Block 2) compared to the initial training session (Block 1) (p < 0.05).

Bilateral Interference in Motor Performance in Homologous vs. Non-homologous Proximal and Distal Effectors

Performance of bimanual motor actions requires coordinated and integrated bilateral communication, but in some bimanual tasks, neural interactions and crosstalk might cause bilateral interference. The level of interference probably depends on the proportions of bilateral interneurons connecting homologous areas of the motor cortex in the two hemispheres. The neuromuscular system for proximal muscles has a higher number of bilateral interneurons connecting homologous areas of the motor cortex compared to distal muscles. Based on the differences in neurophysiological organization for proximal vs. distal effectors in the upper extremities, the purpose of the present experiment was to evaluate how the level of bilateral interference depends on whether the bilateral interference task is performed with homologous or non-homologous effectors as the primary task. Fourteen participants first performed a unilateral primary motor task with the dominant arm with (1) proximal and (2) distal controlled joysticks. Performance in the unilateral condition with the dominant arm was compared to the same effector’s performance when two different bilateral interference tasks were performed simultaneously with the non-dominant arm. The two different bilateral interference tasks were subdivided into (1) homologous and (2) non-homologous effectors. The results showed a significant decrease in performance for both proximal and distal controlled joysticks, and this effect was independent of whether the bilateral interference tasks were introduced with homologous or non-homologous effectors. The overall performance decrease as a result of bilateral interference was larger for proximal compared to distal controlled joysticks. Furthermore, a proximal bilateral interference caused a larger performance decrement independent of whether the primary motor task was controlled by a proximal or distal joystick. A novel finding was that the distal joystick performance equally interfered with either homologous (distal bilateral interference) or non-homologous (proximal bilateral interference) interference tasks performed simultaneously. The results indicate that the proximal–distal distinction is an important organismic constraint on motor control and for understanding bilateral communication and interference in general and, in particular, how bilateral interference caused by homologous vs. non-homologous effectors impacts motor performance for proximal and distal effectors. The results seem to map neuroanatomical and neurophysiological differences for these effectors.

Functional Brain Imaging Reliably Predicts Bimanual Motor Skill Performance in a Standardized Surgical Task

Currently, there is a dearth of objective metrics for assessing bi-manual motor skills, which are critical for high-stakes professions such as surgery. Recently, functional near-infrared spectroscopy (fNIRS) has been shown to be effective at classifying motor task types, which can be potentially used for assessing motor performance level. In this work, we use fNIRS data for predicting the performance scores in a standardized bi-manual motor task used in surgical certification and propose a deep-learning framework 'Brain-NET' to extract features from the fNIRS data. Our results demonstrate that the Brain-NET is able to predict bi-manual surgical motor skills based on neuroimaging data accurately ( R2=0.73). Furthermore, the classification ability of the Brain-NET model is demonstrated based on receiver operating characteristic (ROC) curves and area under the curve (AUC) values of 0.91. Hence, these results establish that fNIRS associated with deep learning analysis is a promising method for a bedside, quick and cost-effective assessment of bi-manual skill levels.

Decreasing the Surgical Errors by Neurostimulation of Primary Motor Cortex and the Associated Brain Activation via Neuroimaging

Acquisition of fine motor skills is a time-consuming process as it is based on learning via frequent repetitions. Transcranial electrical stimulation (tES) is a promising means of enhancing simple motor skill development via neuromodulatory mechanisms. Here, we report that non-invasive neurostimulation facilitates the learning of complex fine bimanual motor skills associated with a surgical task. During the training of 12 medical students on the Fundamentals of Laparoscopic Surgery (FLS) pattern cutting task over a period of 12 days, we observed that transcranial direct current stimulation (tDCS) decreased error level and the variability in performance, compared to the Sham group. Furthermore, by concurrently monitoring the cortical activations of the subjects via functional near-infrared spectroscopy (fNIRS), our study showed that the cortical activation patterns were significantly different between the tDCS and Sham group, with the activation of primary motor cortex (M1) and prefrontal cortex (PFC) contralateral to the anodal electrode significantly decreased while supplemental motor area (SMA) increased by tDCS. The lowered performance errors were retained after 1-month post-training. This work supports the use of tDCS to enhance performance accuracy in fine bimanual motor tasks.

Feasibility of combining functional near-infrared spectroscopy with electroencephalography to identify chronic stroke responders to cerebellar transcranial direct current stimulation-a computational modeling and portable neuroimaging methodological study

Feasibility of portable neuroimaging of cerebellar transcranial direct current stimulation (ctDCS) effects on the cerebral cortex has not been investigated vis-à-vis cerebellar lobular electric field strength. We studied functional near-infrared spectroscopy (fNIRS) in conjunction with electroencephalography (EEG) to measure changes in the brain activation at the prefrontal cortex (PFC) and the sensorimotor cortex (SMC) following ctDCS as well as virtual reality-based balance training (VBaT) before and after ctDCS treatment in 12 hemiparetic chronic stroke survivors. We performed general linear modeling (GLM) that putatively associated the lobular electric field strength with the changes in the fNIRS-EEG measures at the ipsilesional and contra-lesional PFC and SMC. Here, fNIRS-EEG measures were found in the latent space from canonical correlation analysis (CCA) between the changes in total hemoglobin (tHb) concentrations (0.01-0.07Hz and 0.07-0.13Hz bands) and log10-transformed EEG bandpower within 1-45 Hz where significant (Wilks' lambda>0.95) canonical correlations were found only for the 0.07-0.13-Hz band. Also, the first principal component (97.5% variance accounted for) of the mean lobular electric field strength was a good predictor of the latent variables of oxy-hemoglobin (O2Hb) concentrations and log10-transformed EEG bandpower. GLM also provided insights into non-responders to ctDCS who also performed poorly in the VBaT due to ideomotor apraxia. Future studies should investigate fNIRS-EEG joint-imaging in a larger cohort to identify non-responders based on GLM fitting to the fNIRS-EEG data.

High density optical neuroimaging predicts surgeons's subjective experience and skill levels

Measuring cognitive load is important for surgical education and patient safety. Traditional approaches of measuring cognitive load of surgeons utilise behavioural metrics to measure performance and surveys and questionnaires to collect reports of subjective experience. These have disadvantages such as sporadic data, occasionally intrusive methodologies, subjective or misleading self-reporting. In addition, traditional approaches use subjective metrics that cannot distinguish between skill levels. Functional neuroimaging data was collected using a high density, wireless NIRS device from sixteen surgeons (11 attending surgeons and 5 surgery resident) and 17 students while they performed two laparoscopic tasks (Peg transfer and String pass). Participant’s subjective mental load was assessed using the NASA-TLX survey. Machine learning approaches were used for predicting the subjective experience and skill levels. The Prefrontal cortex (PFC) activations were greater in students who reported higher-than-median task load, as measured by the NASA-TLX survey. However in the case of attending surgeons the opposite tendency was observed, namely higher activations in the lower v higher task loaded subjects. We found that response was greater in the left PFC of students particularly near the dorso- and ventrolateral areas. We quantified the ability of PFC activation to predict the differences in skill and task load using machine learning while focussing on the effects of NIRS channel separation distance on the results. Our results showed that the classification of skill level and subjective task load could be predicted based on PFC activation with an accuracy of nearly 90%. Our finding shows that there is sufficient information available in the optical signals to make accurate predictions about the surgeons’ subjective experiences and skill levels. The high accuracy of results is encouraging and suggest the integration of the strategy developed in this study as a promising approach to design automated, more accurate and objective evaluation methods.

Cortical hemodynamics as a function of handgrip strength and cognitive performance: a cross-sectional fNIRS study in younger adults

BACKGROUND

There is growing evidence for a positive correlation between measures of muscular strength and cognitive abilities. However, the neurophysiological correlates of this relationship are not well understood so far. The aim of this study was to investigate cortical hemodynamics [i.e., changes in concentrations of oxygenated (oxyHb) and deoxygenated hemoglobin (deoxyHb)] as a possible link between measures of muscular strength and cognitive performance.

METHODS

In a cohort of younger adults (n = 39, 18-30 years), we assessed (i) handgrip strength by a handhold dynamometer, (ii) short-term working memory performance by using error rates and reaction times in the Sternberg task, and (iii) cortical hemodynamics of the prefrontal cortex (PFC) via functional near-infrared spectroscopy (fNIRS).

RESULTS

We observed low to moderate negative correlations (rp =  ~ - 0.38 to - 0.51; p < 0.05) between reaction time and levels of oxyHb in specific parts of the PFC. Furthermore, we noticed low to moderate positive correlations (rp =  ~ 0.34 to 0.45; p < 0.05) between reaction times and levels of deoxyHb in distinct parts of the PFC. Additionally, higher levels of oxyHb (rp (35) = 0.401; p = 0.014) and lower levels of deoxyHb (rp (34) = - 0.338; p = 0.043) in specific parts of the PFC were linked to higher percentage of correct answers. We also found low to moderate correlations (p < 0.05) between measures of handgrip strength and levels of oxyHb (rp =  ~ 0.35; p < 0.05) and levels of deoxyHb (rp =  ~ - 0.25 to - 0.49; p < 0.05) in specific parts of the PFC. However, there was neither a correlation between cognitive performance and handgrip strength nor did cortical hemodynamics in the PFC mediate the relationship between handgrip strength and cognitive performance (p > 0.05).

CONCLUSION

The present study provides evidence for a positive neurobehavioral relationship between cortical hemodynamics and cognitive performance. Our findings further imply that in younger adults higher levels of handgrip strength positively influence cortical hemodynamics although the latter did not necessarily culminate in better cognitive performance. Future research should examine whether the present findings can be generalized to other cohorts (e.g., older adults).

Functional brain connectivity related to surgical skill dexterity in physical and virtual simulation environments

Significance: Surgical simulators, both virtual and physical, are increasingly used as training tools for teaching and assessing surgical technical skills. However, the metrics used for assessment in these simulation environments are often subjective and inconsistent. Aim: We propose functional activation metrics, derived from brain imaging measurements, to objectively assess the correspondence between brain activation with surgical motor skills for subjects with varying degrees of surgical skill. Approach: Cortical activation based on changes in the oxygenated hemoglobin (HbO) of 36 subjects was measured using functional near-infrared spectroscopy at the prefrontal cortex (PFC), primary motor cortex, and supplementary motor area (SMA) due to their association with motor skill learning. Inter-regional functional connectivity metrics, namely, wavelet coherence (WCO) and wavelet phase coherence were derived from HbO changes to correlate brain activity to surgical motor skill levels objectively. Results: One-way multivariate analysis of variance found a statistically significant difference in the inter-regional WCO metrics for physical simulator based on Wilk's Λ for expert versus novice, F ( 10,1 ) = 7495.5 , p < 0.01 . Partial eta squared effect size for the inter-regional WCO metrics was found to be highest between the central prefrontal cortex (CPFC) and SMA, CPFC-SMA ( η 2 = 0.257 ). Two-tailed Mann-Whitney U tests with a 95% confidence interval showed baseline equivalence and a statistically significant ( p < 0.001 ) difference in the CPFC-SMA WPCO metrics for the physical simulator training group ( 0.960 ± 0.045 ) versus the untrained control group ( 0.735 ± 0.177 ) following training for 10 consecutive days in addition to the pretest and posttest days. Conclusion: We show that brain functional connectivity WCO metric corresponds to surgical motor skills in the laparoscopic physical simulators. Functional connectivity between the CPFC and the SMA is lower for subjects that exhibit expert surgical motor skills than untrained subjects in laparoscopic physical simulators.

How Wearable Technology Can Facilitate AI Analysis of Surgical Videos

Operative video has great potential to enable instant replays of critical surgical decisions for training and quality review. Recently, artificial intelligence (AI) has shown early promise as a method of enabling efficient video review, analysis, and segmentation. Despite the progress with AI analysis of surgical videos, more work needs to be done to improve the accuracy and efficiency of AI-driven video analysis. At a recent consensus conference held on July 10-11, 2020, 8 research teams shared their work using AI for surgical video analysis. Four of the teams showcased the utility of wearable technology in providing objective surgical metrics. Data from these technologies were shown to pinpoint important cognitive and motor actions during operative tasks and procedures. The results support the utility of wearable technology to facilitate efficient and accurate video analysis and segmentation.

More Pronounced Bimanual Interference in Proximal Compared to Distal Effectors of the Upper Extremities

Bimanual performance depends on effective and modular bilateral communication between the two bodysides. Bilateral neural interactions between the bodysides could cause bimanual interference, and the neuromuscular system for proximal and distal muscles is differently organized, where proximal muscles have more bilateral interneurons at both cortical and spinal level compared to distal muscles. These differences might increase the potential for bimanual interference between proximal arm muscles, because of greater proportions of bilateral interneurons to proximal muscles. The purpose of the present experiment was to evaluate potential differences in bimanual interference between proximal versus distal effectors in the upper extremities. 14 participants first performed a unilateral primary motor task with dominant arm with (1) a proximal and (2) distal controlled joysticks (condition A). Performance in condition A, was compared with the same effector’s performance when a bimanual interference task was performed simultaneously with the non-dominant arm (condition B). The results showed a significant bimanual interference for both the proximal and distal controlled joysticks. Most interestingly, the bimanual interference was larger for the proximal joystick compared to the distal controlled joystick. The increase in spatial accuracy error was higher for the proximal controlled joystick, compared with the distal controlled joystick. These results indicate that the proximal-distal distinction is an important organismic constraint on motor control, and especially for bilateral communication. There seem to be an undesired bilateral interference for both proximal and distal muscles. The interference is higher in the case of proximal effectors compared distal effectors, and the results seem to map the neuroanatomical and neurophysiological differences for these effectors.

Cortical Activation During Shoulder and Finger Movements in Healthy Adults: A Functional Near-Infrared Spectroscopy (fNIRS) Study

Characterization of cortical activation patterns during movement of the upper extremity in healthy adults is helpful in understanding recovery mechanisms following neurological disorders. This study explores cortical activation patterns associated with movements of the shoulder and fingers in healthy adults using functional near-infrared spectroscopy (fNIRS). Twelve healthy right-handed participants were recruited. Two motor tasks (shoulder abduction and finger extension) with two different trial lengths (10 s and 20 s) were performed in a sitting position at a rate of 0.5 Hz. The hemodynamic response, as indicated by oxy-hemoglobin (HbO) and deoxy-hemoglobin (HbR), over both hemispheres was acquired using a 54-channel fNIRS system. We found a generalized bilateral cortical activation during both motor tasks with greater activation in the contralateral compared to the ipsilateral primary motor cortex. Particularly in the more medial part of the contralateral hemisphere, significant higher activation was found during the shoulder compared to finger movements. Furthermore, cortical activation patterns are affected not only by motor tasks but also by trial lengths. HbO is more sensitive to detect cortical activation during finger movements in longer trials, while HbR is a better surrogate to capture active areas during shoulder movement in shorter trials. Based on these findings, reporting both HbO and HbR is strongly recommended for future fNIRS studies, and trial lengths should be taken into account when designing experiments and explaining results. Our findings demonstrating distinct cortical activation patterns associated with shoulder and finger movements in healthy adults provide a foundation for future research to study recovery mechanisms following neurological disorders.

Neuroenhancement in surgeons: benefits, risks and ethical dilemmas

Background

Surgeons traditionally aim to reduce mistakes in healthcare through repeated training and advancement of surgical technology. Recently, performance-enhancing interventions such as neurostimulation are emerging which may offset errors in surgical practice.

Methods

Use of transcranial direct-current stimulation (tDCS), a novel neuroenhancement technique that has been applied to surgeons to improve surgical technical performance, was reviewed. Evidence supporting tDCS improvements in motor and cognitive performance outside of the field of surgery was assessed and correlated with emerging research investigating tDCS in the surgical setting and potential applications to wider aspects of healthcare. Ethical considerations and future implications of using tDCS in surgical training and perioperatively are also discussed.

Results

Outside of surgery, tDCS studies demonstrate improved motor performance with regards to reaction time, task completion, strength and fatigue, while also suggesting enhanced cognitive function through multitasking, vigilance and attention assessments. In surgery, current research has demonstrated improved performance in open knot-tying, laparoscopic and robotic skills while also offsetting subjective temporal demands. However, a number of ethical issues arise from the potential application of tDCS in surgery in the form of safety, coercion, distributive justice and fairness, all of which must be considered prior to implementation.

Conclusion

Neuroenhancement may improve motor and cognitive skills in healthcare professions with impact on patient safety. Implementation will require accurate protocols and regulations to balance benefits with the associated ethical dilemmas, and to direct safe use for clinicians and patients.

Training with cognitive load improves performance under similar conditions in a real surgical task

BACKGROUND

Enhancing cognitive load while performing a bimanual surgical task affects performance. Whether repeated training under this condition could benefit performance in an operating room was tested using a virtual reality simulator with cognitive load applied through two-digit math multiplication questions.

METHOD

11 subjects were randomized to Control, VR and VR + CL groups. After a pre-test, VR and VR + CL groups repeated the peg transfer task 150 times over 15 sessions with cognitive load applied only for the last 100 trials. After training, all groups took a post-test and two weeks later the retention test with and without cognitive load and the transfer task on a pig intestine of 150 cm long under cognitive load.

RESULTS AND CONCLUSION

Mixed ANOVA analysis showed significant differences between the control and VR and VR + CL groups (p = 0.013, p = 0.009) but no differences between the VR + CL and the VR groups (p = 1.0). GOALS bimanual dexterity score on transfer test show that VR + CL group outperformed both Control and VR groups (p = 0.016, p = 0.03). Training under cognitive load benefitted performance on an actual surgical task under similar conditions.

Neurophysiological changes associated with training in laparoscopic surgery using EEG: a pilot study

Laparoscopy is a minimally invasive technique that requires surgeons to acquire special motor skills derived from an extensive training. This work focuses on exploring the neurophysiological changes associated with motor learning. Electroencephalographic (EEG) signals were recorded from eight subjects while performing a bimanual coordination task in a laparoscopic simulator. Spectral power measurements in theta, alpha and beta bands during four training sessions were calculated. Power indices, task score and perception of mental workload were evaluated using analysis of variance to show the effect of training session. Results show improvements in task performance and changes in power measurements associated with the training process. This work opens the possibility to assess the training performance of surgical residents using electrophysiological recordings.

Performance Monitoring via Functional Near Infrared Spectroscopy for Virtual Reality Based Basic Life Support Training

The use of serious game tools in training of medical professions is steadily growing. However, there is a lack of reliable performance assessment methods to evaluate learner’s outcome. The aim of this study is to determine whether functional near infrared spectroscopy (fNIRS) can be used as an additional tool for assessing the learning outcome of virtual reality (VR) based learning modules. The hypothesis is that together with an improvement in learning outcome there would be a decrease in the participants’ cerebral oxygenation levels measured from the prefrontal cortex (PFC) region and an increase of participants’ serious gaming results. To test this hypothesis, the subjects were recruited and divided into four groups with different combinations of prior virtual reality experience and prior Basic Life Support (BLS) knowledge levels. A VR based serious gaming module for teaching BLS and 16-Channel fNIRS system were used to collect data from the participants. Results of the participants’ scores acquired from the serious gaming module were compared with fNIRS measures on the initial and final training sessions. Kruskal Wallis test was run to determine any significant statistical difference between the groups and Mann–Whitney U test was utilized to obtain pairwise comparisons. BLS training scores of the participants acquired from VR based serious game’s the learning management system and fNIRS measurements revealed decrease in use of resources from the PFC, but increase in behavioral performance. Importantly, brain-based measures can provide an additional quantitative metric for trainee’s expertise development and can assist the medical simulation instructors.

A machine learning approach to predict surgical learning curves

BACKGROUND

Contemporary surgical training programs rely on the repetition of selected surgical motor tasks. Such methodology is inherently open ended with no control on the time taken to attain a set level of proficiency, given the trainees' intrinsic differences in initial skill levels and learning abilities. Hence, an efficient training program should aim at tailoring the surgical training protocols to each trainee. In this regard, a predictive model using information from the initial learning stage to predict learning curve characteristics should facilitate the whole surgical training process.

METHODS

This paper analyzes learning curve data to train a multivariate supervised machine learning model. One factor is extracted to define the trainees' learning ability. An unsupervised machine learning model is also utilized for trainee classification. When established, the model can predict robustly the learning curve characteristics based on the first few trials.

RESULTS

We show that the information present in the first 10 trials of surgical tasks can be utilized to predict the number of trials required to achieve proficiency (R²=0.72) and the final performance level (R²=0.89). Furthermore, only a single factor, learning index, is required to describe the learning process and to classify learners with unique learning characteristics.

CONCLUSION

Using machine learning models, we show, for the first time, that the first few trials contain sufficient information to predict learning curve characteristics and that a single factor can capture the complex learning behavior. Using such models holds the potential for personalization of training regimens, leading to greater efficiency and lower costs.

Towards a Multimodal Model of Cognitive Workload Through Synchronous Optical Brain Imaging and Eye Tracking Measures

Recent advances in neuroimaging technologies have rendered multimodal analysis of operators’ cognitive processes in complex task settings and environments increasingly more practical. In this exploratory study, we utilized optical brain imaging and mobile eye tracking technologies to investigate the behavioral and neurophysiological differences among expert and novice operators while they operated a human-machine interface in normal and adverse conditions. In congruence with related work, we observed that experts tended to have lower prefrontal oxygenation and exhibit gaze patterns that are better aligned with the optimal task sequence with shorter fixation durations as compared to novices. These trends reached statistical significance only in the adverse condition where the operators were prompted with an unexpected error message. Comparisons between hemodynamic and gaze measures before and after the error message indicated that experts’ neurophysiological response to the error involved a systematic increase in bilateral dorsolateral prefrontal cortex (dlPFC) activity accompanied with an increase in fixation durations, which suggests a shift in their attentional state, possibly from routine process execution to problem detection and resolution. The novices’ response was not as strong as that of experts, including a slight increase only in the left dlPFC with a decreasing trend in fixation durations, which is indicative of visual search behavior for possible cues to make sense of the unanticipated situation. A linear discriminant analysis model capitalizing on the covariance structure among hemodynamic and eye movement measures could distinguish experts from novices with 91% accuracy. Despite the small sample size, the performance of the linear discriminant analysis combining eye fixation and dorsolateral oxygenation measures before and after an unexpected event suggests that multimodal approaches may be fruitful for distinguishing novice and expert performance in similar neuroergonomic applications in the field.