Mental workload during n-back task-quantified in the prefrontal cortex using fNIRS

Differential Entropy Preserves Variational Information of Near-Infrared Spectroscopy Time Series Associated With Working Memory

Neuroscience research shows a growing interest in the application of Near-Infrared Spectroscopy (NIRS) in analysis and decoding of the brain activity of human subjects. Given the correlation that is observed between the Blood Oxygen Dependent Level (BOLD) responses that are exhibited by the time series data of functional Magnetic Resonance Imaging (fMRI) and the hemoglobin oxy/deoxy-genation that is captured by NIRS, linear models play a central role in these applications. This, in turn, results in adaptation of the feature extraction strategies that are well-suited for discretization of data that exhibit a high degree of linearity, namely, slope and the mean as well as their combination, to summarize the informational contents of the NIRS time series. In this article, we demonstrate that these features are inefficient in capturing the variational information of NIRS data, limiting the reliability and the adequacy of the conclusion on their results. Alternatively, we propose the linear estimate of differential entropy of these time series as a natural representation of such information. We provide evidence for our claim through comparative analysis of the application of these features on NIRS data pertinent to several working memory tasks as well as naturalistic conversational stimuli.

Exploring the role of task performance and learning style on prefrontal hemodynamics during a working memory task

Existing literature outlines the quality and location of activation in the prefrontal cortex (PFC) during working memory (WM) tasks. However, the effects of individual differences on the underlying neural process of WM tasks are still unclear. In this functional near infrared spectroscopy study, we administered a visual and auditory n-back task to examine activation in the PFC while considering the influences of task performance, and preferred learning strategy (VARK score). While controlling for age, results indicated that high performance (HP) subjects (accuracy > 90%) showed task dependent lower activation compared to normal performance subjects in PFC region Specifically HP groups showed lower activation in left dorsolateral PFC (DLPFC) region during performance of auditory task whereas during visual task they showed lower activation in the right DLPFC. After accounting for learning style, we found a correlation between visual and aural VARK score and level of activation in the PFC. Subjects with higher visual VARK scores displayed lower activation during auditory task in left DLPFC, while those with higher visual scores exhibited higher activation during visual task in bilateral DLPFC. During performance of auditory task, HP subjects had higher visual VARK scores compared to NP subjects indicating an effect of learning style on the task performance and activation. The results of this study show that learning style and task performance can influence PFC activation, with applications toward neurological implications of learning style and populations with deficits in auditory or visual processing.

In silico vs. Over the Clouds: On-the-Fly Mental State Estimation of Aircraft Pilots, Using a Functional Near Infrared Spectroscopy Based Passive-BCI

There is growing interest for implementing tools to monitor cognitive performance in naturalistic work and everyday life settings. The emerging field of research, known as neuroergonomics, promotes the use of wearable and portable brain monitoring sensors such as functional near infrared spectroscopy (fNIRS) to investigate cortical activity in a variety of human tasks out of the laboratory. The objective of this study was to implement an on-line passive fNIRS-based brain computer interface to discriminate two levels of working memory load during highly ecological aircraft piloting tasks. Twenty eight recruited pilots were equally split into two groups (flight simulator vs. real aircraft). In both cases, identical approaches and experimental stimuli were used (serial memorization task, consisting in repeating series of pre-recorded air traffic control instructions, easy vs. hard). The results show pilots in the real flight condition committed more errors and had higher anterior prefrontal cortex activation than pilots in the simulator, when completing cognitively demanding tasks. Nevertheless, evaluation of single trial working memory load classification showed high accuracy (>76%) across both experimental conditions. The contributions here are two-fold. First, we demonstrate the feasibility of passively monitoring cognitive load in a realistic and complex situation (live piloting of an aircraft). In addition, the differences in performance and brain activity between the two experimental conditions underscore the need for ecologically-valid investigations.

Frontal Underactivation During Working Memory Processing in Adults With Acute Partial Sleep Deprivation: A Near-Infrared Spectroscopy Study

Individuals with partial sleep deprivation may have working memory (WM) impairment, but the underlying neural mechanism of this phenomenon is relatively unknown. The present study examined neural processing during WM performance in individuals with and without partial sleep deprivation using near-infrared spectroscopy (NIRS). Forty college students (10 males) were equally split into Sufficient Sleep (SS) and Insufficient Sleep (IS) groups based on self-reports of previous night's sleep duration. Participants in the SS group obtained the recommended amounts of sleep according to various sleep organizations (i.e., >7.0 h), whereas those in the IS group obtained amounts of sleep no greater than the lower limit of the recommendation (i.e., ≤7.0 h). All participants underwent an n-back paradigm with a WM load (i.e., 3-back) and a control condition (i.e., 0-back) while their prefrontal hemodynamics were recorded by NIRS. The IS and SS groups performed the tasks comparably well. However, unlike the SS group, which exhibited bilateral frontal activation indicated by increased oxyhemoglobin concentration and decreased deoxyhemoglobin concentration during WM processing (i.e., 3-back > 0-back), the IS group did not exhibit such activation. In addition, levels of WM-related frontal activation, especially those on the left side, correlated with sleep duration the night before, even when habitual sleep duration was controlled for. The findings suggest the presence of frontal lobe dysfunction in the absence of evident WM difficulties in individuals with acute partial sleep deprivation. They also highlight the importance of a good night's sleep to brain health.

Eyes-closed hybrid brain-computer interface employing frontal brain activation

Brain-computer interfaces (BCIs) have been studied extensively in order to establish a non-muscular communication channel mainly for patients with impaired motor functions. However, many limitations remain for BCIs in clinical use. In this study, we propose a hybrid BCI that is based on only frontal brain areas and can be operated in an eyes-closed state for end users with impaired motor and declining visual functions. In our experiment, electroencephalography (EEG) and near-infrared spectroscopy (NIRS) were simultaneously measured while 12 participants performed mental arithmetic (MA) and remained relaxed (baseline state: BL). To evaluate the feasibility of the hybrid BCI, we classified MA- from BL-related brain activation. We then compared classification accuracies using two unimodal BCIs (EEG and NIRS) and the hybrid BCI in an offline mode. The classification accuracy of the hybrid BCI (83.9 ± 10.3%) was shown to be significantly higher than those of unimodal EEG-based (77.3 ± 15.9%) and NIRS-based BCI (75.9 ± 6.3%). The analytical results confirmed performance improvement with the hybrid BCI, particularly for only frontal brain areas. Our study shows that an eyes-closed hybrid BCI approach based on frontal areas could be applied to neurodegenerative patients who lost their motor functions, including oculomotor functions.

The Validity of Functional Near-Infrared Spectroscopy Recordings of Visuospatial Working Memory Processes in Humans

Functional near infrared spectroscopy (fNIRS) is increasingly used for investigating cognitive processes. To provide converging evidence for the validity of fNIRS recordings in cognitive neuroscience, we investigated functional activation in the frontal cortex in 43 participants during the processing of a visuospatial working memory (WM) task and a sensory duration discrimination (DD) task functionally unrelated to WM. To distinguish WM-related processes from a general effect of increased task demand, we applied an adaptive approach, which ensured that subjective task demand was virtually identical for all individuals and across both tasks. Our specified region of interest covered Brodmann Area 8 of the left hemisphere, known for its important role in the execution of WM processes. Functional activation, as indicated by an increase of oxygenated and a decrease of deoxygenated hemoglobin, was shown for the WM task, but not in the DD task. The overall pattern of results indicated that hemodynamic responses recorded by fNIRS are sensitive to specific visuospatial WM capacity-related processes and do not reflect a general effect of increased task demand. In addition, the finding that no such functional activation could be shown for participants with far above-average mental ability suggested different cognitive processes adopted by this latter group.

Detecting Pilot's Engagement Using fNIRS Connectivity Features in an Automated vs. Manual Landing Scenario

Monitoring pilot's mental states is a relevant approach to mitigate human error and enhance human machine interaction. A promising brain imaging technique to perform such a continuous measure of human mental state under ecological settings is Functional Near-InfraRed Spectroscopy (fNIRS). However, to our knowledge no study has yet assessed the potential of fNIRS connectivity metrics as long as passive Brain Computer Interfaces (BCI) are concerned. Therefore, we designed an experimental scenario in a realistic simulator in which 12 pilots had to perform landings under two contrasted levels of engagement (manual vs. automated). The collected data were used to benchmark the performance of classical oxygenation features (i.e., Average, Peak, Variance, Skewness, Kurtosis, Area Under the Curve, and Slope) and connectivity features (i.e., Covariance, Pearson's, and Spearman's Correlation, Spectral Coherence, and Wavelet Coherence) to discriminate these two landing conditions. Classification performance was obtained by using a shrinkage Linear Discriminant Analysis (sLDA) and a stratified cross validation using each feature alone or by combining them. Our findings disclosed that the connectivity features performed significantly better than the classical concentration metrics with a higher accuracy for the wavelet coherence (average: 65.3/59.9 %, min: 45.3/45.0, max: 80.5/74.7 computed for HbO/HbR signals respectively). A maximum classification performance was obtained by combining the area under the curve with the wavelet coherence (average: 66.9/61.6 %, min: 57.3/44.8, max: 80.0/81.3 computed for HbO/HbR signals respectively). In a general manner all connectivity measures allowed an efficient classification when computed over HbO signals. Those promising results provide methodological cues for further implementation of fNIRS-based passive BCIs.

Reduced Frontal Activations at High Working Memory Load in Mild Cognitive Impairment: Near-Infrared Spectroscopy

BACKGROUND

Some functional magnetic resonance imaging studies have reported altered activations in the frontal cortex during working memory (WM) performance in individuals with mild cognitive impairment (MCI), but the findings have been mixed. The objective of the present study was to utilize near-infrared spectroscopy (NIRS), an alternative imaging technique, to examine neural processing during WM performance in individuals with MCI.

METHODS

Twenty-six older adults with MCI (7 males; mean age 69.15 years) were compared with 26 age-, gender-, handedness-, and education-matched older adults with normal cognition (NC; 7 males; mean age 68.87 years). All of the participants undertook an n-back task with a low (i.e., 0-back) and a high (i.e., 2-back) WM load condition while their prefrontal dynamics were recorded by a 16-channel NIRS system.

RESULTS

Although behavioral results showed that the two groups had comparable task performance, neuroimaging results showed that the MCI group, unlike the NC group, did not exhibit significantly increased frontal activations bilaterally when WM load increased. Compared to the NC group, the MCI group had similar frontal activations at low load (p > 0.05 on all channels) but reduced activations at high load (p < 0.05 on 4 channels), thus failing to demonstrate WM-related frontal activations (p < 0.05 on 9 channels). In addition, we found a positive correlation between the left WM-related frontal activations and WM ability primarily in the NC group (rs = 0.42, p = 0.035), suggesting a relationship between frontal hypoactivation and WM difficulties.

CONCLUSION

The present findings suggest the presence of frontal dysfunction that is dependent on WM load in individuals with MCI.

Listening in Naturalistic Scenes: What Can Functional Near-Infrared Spectroscopy and Intersubject Correlation Analysis Tell Us About the Underlying Brain Activity?

Listening to speech in the noisy conditions of everyday life can be effortful, reflecting the increased cognitive workload involved in extracting meaning from a degraded acoustic signal. Studying the underlying neural processes has the potential to provide mechanistic insight into why listening is effortful under certain conditions. In a move toward studying listening effort under ecologically relevant conditions, we used the silent and flexible neuroimaging technique functional near-infrared spectroscopy (fNIRS) to examine brain activity during attentive listening to speech in naturalistic scenes. Thirty normally hearing participants listened to a series of narratives continuously varying in acoustic difficulty while undergoing fNIRS imaging. Participants then listened to another set of closely matched narratives and rated perceived effort and intelligibility for each scene. As expected, self-reported effort generally increased with worsening signal-to-noise ratio. After controlling for better-ear signal-to-noise ratio, perceived effort was greater in scenes that contained competing speech than in those that did not, potentially reflecting an additional cognitive cost of overcoming informational masking. We analyzed the fNIRS data using intersubject correlation, a data-driven approach suitable for analyzing data collected under naturalistic conditions. Significant intersubject correlation was seen in the bilateral auditory cortices and in a range of channels across the prefrontal cortex. The involvement of prefrontal regions is consistent with the notion that higher order cognitive processes are engaged during attentive listening to speech in complex real-world conditions. However, further research is needed to elucidate the relationship between perceived listening effort and activity in these extended cortical networks.

Development of a Ternary Near-Infrared Spectroscopy Brain-Computer Interface: Online Classification of Verbal Fluency Task, Stroop Task and Rest

The majority of proposed NIRS-BCIs has considered binary classification. Studies considering high-order classification problems have yielded average accuracies that are less than favorable for practical communication. Consequently, there is a paucity of evidence supporting online classification of more than two mental states using NIRS. We developed an online ternary NIRS-BCI that supports the verbal fluency task (VFT), Stroop task and rest. The system utilized two sessions dedicated solely to classifier training. Additionally, samples were collected prior to each period of online classification to update the classifier. Using a continuous-wave spectrometer, measurements were collected from the prefrontal and parietal cortices while 11 able-bodied adult participants were cued to perform one of the two cognitive tasks or rests. Each task was used to indicate the desire to select a particular letter on a scanning interface, while rest avoided selection. Classification was performed using 25 iteration of bagging with a linear discriminant base classifier. Classifiers were trained on 10-dimensional feature sets. The BCI's classification decision was provided as feedback. An average online classification accuracy of [Formula: see text]% was achieved, representing an ITR of [Formula: see text] bits/min. The results demonstrate that online communication can be achieved with a ternary NIRS-BCI that supports VFT, Stroop task and rest. Our findings encourage continued efforts to enhance the ITR of NIRS-BCIs.

Utilizing functional near-infrared spectroscopy for prediction of cognitive workload in noisy work environments

Occupational noise frequently occurs in the work environment in military intelligence, surveillance, and reconnaissance operations. This impacts cognitive performance by acting as a stressor, potentially interfering with the analysts' decision-making process. We investigated the effects of different noise stimuli on analysts' performance and workload in anomaly detection by simulating a noisy work environment. We utilized functional near-infrared spectroscopy (fNIRS) to quantify oxy-hemoglobin (HbO) and deoxy-hemoglobin concentration changes in the prefrontal cortex (PFC), as well as behavioral measures, which include eye tracking, reaction time, and accuracy rate. We hypothesized that noisy environments would have a negative effect on the participant in terms of anomaly detection performance due to the increase in workload, which would be reflected by an increase in PFC activity. We found that HbO for some of the channels analyzed were significantly different across noise types ([Formula: see text]). Our results also indicated that HbO activation for short-intermittent noise stimuli was greater in the PFC compared to long-intermittent noises. These approaches using fNIRS in conjunction with an understanding of the impact on human analysts in anomaly detection could potentially lead to better performance by optimizing work environments.

Increased prefrontal cortex connectivity during cognitive challenge assessed by fNIRS imaging

In this study, functional near-infrared spectroscopy (fNIRS) and the graph theory approach were used to access the functional connectivity (FC) of the prefrontal cortex (PFC) in a resting state and during increased mental workload. For this very purpose, a pattern recognition-based test was developed, which elicited a strong response throughout the PFC during the test condition. FC parameters obtained during stimulation were found increased compared to those in a resting state after correlation based signal improvement (CBSI), which can attenuate those components of fNIRS signals which are unrelated to neural activity. These results indicate that the cognitive challenge increased the FC in the PFC and suggests a great potential in investigating FC in various cognitive states.

Development and testing an online near-infrared spectroscopy brain-computer interface tailored to an individual with severe congenital motor impairments

PURPOSE

For non-verbal individuals, brain-computer interfaces (BCIs) are a potential means of communication. Near-infrared spectroscopy (NIRS) is a brain-monitoring modality that has been considered for BCIs. To date, limited NIRS-BCI testing has involved online classification, particularly with individuals with severe motor impairments.

MATERIALS AND METHODS

We tested an online NIRS-BCI developed for a non-verbal individual with severe congenital motor impairments. The binary BCI differentiated categorical verbal fluency task (VFT) performance and rest using prefrontal measurements. The participant attended five sessions, the last two of which were online with classification feedback.

RESULTS

An online classification accuracy of 63.33% was achieved using a linear discriminant classifier trained on a four-dimensional feature set. An offline, cross-validation analysis of all data yielded an optimal adjusted classification accuracy of 66.6 ± 9.11%. Inconsistent functional responses, contradictory effects of feedback, participant fatigue and motion artefacts were identified as challenges to online classification specific to this participant.

CONCLUSIONS

Results suggest potential in using an NIRS-BCI controlled by the VFT in instances of severe congenital impairments. Further testing with users with severe disabilities is necessary. Implications for Rehabilitation Brain-computer interfaces (BCIs) can provide a non-motor based means of communication for individuals with severe motor impairments. Near-infrared spectroscopy (NIRS) is a haemodynamic-based brain-imaging modality used in BCIs. To date, NIRS-BCIs have not been thoroughly tested with potential target users. This case study shows that NIRS-BCIs may offer a means of practical communication for individuals with severe congenital impairments and continued exploration is advisable.

Multisubject "Learning" for Mental Workload Classification Using Concurrent EEG, fNIRS, and Physiological Measures

An accurate measure of mental workload level has diverse neuroergonomic applications ranging from brain computer interfacing to improving the efficiency of human operators. In this study, we integrated electroencephalogram (EEG), functional near-infrared spectroscopy (fNIRS), and physiological measures for the classification of three workload levels in an n-back working memory task. A significantly better than chance level classification was achieved by EEG-alone, fNIRS-alone, physiological alone, and EEG+fNIRS based approaches. The results confirmed our previous finding that integrating EEG and fNIRS significantly improved workload classification compared to using EEG-alone or fNIRS-alone. The inclusion of physiological measures, however, does not significantly improves EEG-based or fNIRS-based workload classification. A major limitation of currently available mental workload assessment approaches is the requirement to record lengthy calibration data from the target subject to train workload classifiers. We show that by learning from the data of other subjects, workload classification accuracy can be improved especially when the amount of data from the target subject is small.

Measuring Mental Workload with EEG+fNIRS

We studied the capability of a Hybrid functional neuroimaging technique to quantify human mental workload (MWL). We have used electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) as imaging modalities with 17 healthy subjects performing the letter n-back task, a standard experimental paradigm related to working memory (WM). The level of MWL was parametrically changed by variation of n from 0 to 3. Nineteen EEG channels were covering the whole-head and 19 fNIRS channels were located on the forehead to cover the most dominant brain region involved in WM. Grand block averaging of recorded signals revealed specific behaviors of oxygenated-hemoglobin level during changes in the level of MWL. A machine learning approach has been utilized for detection of the level of MWL. We extracted different features from EEG, fNIRS, and EEG+fNIRS signals as the biomarkers of MWL and fed them to a linear support vector machine (SVM) as train and test sets. These features were selected based on their sensitivity to the changes in the level of MWL according to the literature. We introduced a new category of features within fNIRS and EEG+fNIRS systems. In addition, the performance level of each feature category was systematically assessed. We also assessed the effect of number of features and window size in classification performance. SVM classifier used in order to discriminate between different combinations of cognitive states from binary- and multi-class states. In addition to the cross-validated performance level of the classifier other metrics such as sensitivity, specificity, and predictive values were calculated for a comprehensive assessment of the classification system. The Hybrid (EEG+fNIRS) system had an accuracy that was significantly higher than that of either EEG or fNIRS. Our results suggest that EEG+fNIRS features combined with a classifier are capable of robustly discriminating among various levels of MWL. Results suggest that EEG+fNIRS should be preferred to only EEG or fNIRS, in developing passive BCIs and other applications which need to monitor users' MWL.

Mental workload and neural efficiency quantified in the prefrontal cortex using fNIRS

An improved understanding of how the brain allocates mental resources as a function of task difficulty is critical for enhancing human performance. Functional near infrared spectroscopy (fNIRS) is a field-deployable optical brain monitoring technology that provides a direct measure of cerebral blood flow in response to cognitive activity. We found that fNIRS was sensitive to variations in task difficulty in both real-life (flight simulator) and laboratory settings (tests measuring executive functions), showing increased concentration of oxygenated hemoglobin (HbO2) and decreased concentration of deoxygenated hemoglobin (HHb) in the prefrontal cortex as the tasks became more complex. Intensity of prefrontal activation (HbO2 concentration) was not clearly correlated to task performance. Rather, activation intensity shed insight on the level of mental effort, i.e., how hard an individual was working to accomplish a task. When combined with performance, fNIRS provided an estimate of the participants' neural efficiency, and this efficiency was consistent across levels of difficulty of the same task. Overall, our data support the suitability of fNIRS to assess the mental effort related to human operations and represents a promising tool for the measurement of neural efficiency in other contexts such as training programs or the clinical setting.

Assessment of mental workload by EEG+FNIRS

We investigated the use of a multimodal functional neuroimaging system in quantifying mental workload of healthy human volunteers. We recorded behavioral performance measures as well as electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) simultaneously from subjects performing n-back tasks. The EEG and fNIRS signals were used in feature generation and classification offline using support vector machines. We examined the classification accuracy of three distinct systems: EEG based; fNIRS based; and Hybrid, which contained features from the first two systems as based on their interactions. The classification accuracy of the Hybrid system was observed to be greater than that of either system, indicating the synergistic role played by multimodal signals and by neurovascular coupling in quantifying mental workload.

Assessing the Driver's Current Level of Working Memory Load with High Density Functional Near-infrared Spectroscopy: A Realistic Driving Simulator Study

Cognitive overload or underload results in a decrease in human performance which may result in fatal incidents while driving. We envision that driver assistive systems which adapt their functionality to the driver's cognitive state could be a promising approach to reduce road accidents due to human errors. This research attempts to predict variations of cognitive working memory load levels in a natural driving scenario with multiple parallel tasks and to reveal predictive brain areas. We used a modified version of the n-back task to induce five different working memory load levels (from 0-back up to 4-back) forcing the participants to continuously update, memorize, and recall the previous 'n' speed sequences and adjust their speed accordingly while they drove for approximately 60 min on a highway with concurrent traffic in a virtual reality driving simulator. We measured brain activation using multichannel whole head, high density functional near-infrared spectroscopy (fNIRS) and predicted working memory load level from the fNIRS data by combining multivariate lasso regression and cross-validation. This allowed us to predict variations in working memory load in a continuous time-resolved manner with mean Pearson correlations between induced and predicted working memory load over 15 participants of 0.61 [standard error (SE) 0.04] and a maximum of 0.8. Restricting the analysis to prefrontal sensors placed over the forehead reduced the mean correlation to 0.38 (SE 0.04), indicating additional information gained through whole head coverage. Moreover, working memory load predictions derived from peripheral heart rate parameters achieved much lower correlations (mean 0.21, SE 0.1). Importantly, whole head fNIRS sampling revealed increasing brain activation in bilateral inferior frontal and bilateral temporo-occipital brain areas with increasing working memory load levels suggesting that these areas are specifically involved in workload-related processing.

A Non-parametric Approach to the Overall Estimate of Cognitive Load Using NIRS Time Series

We present a non-parametric approach to prediction of the n-back n ∈ {1, 2} task as a proxy measure of mental workload using Near Infrared Spectroscopy (NIRS) data. In particular, we focus on measuring the mental workload through hemodynamic responses in the brain induced by these tasks, thereby realizing the potential that they can offer for their detection in real world scenarios (e.g., difficulty of a conversation). Our approach takes advantage of intrinsic linearity that is inherent in the components of the NIRS time series to adopt a one-step regression strategy. We demonstrate the correctness of our approach through its mathematical analysis. Furthermore, we study the performance of our model in an inter-subject setting in contrast with state-of-the-art techniques in the literature to show a significant improvement on prediction of these tasks (82.50 and 86.40% for female and male participants, respectively). Moreover, our empirical analysis suggest a gender difference effect on the performance of the classifiers (with male data exhibiting a higher non-linearity) along with the left-lateralized activation in both genders with higher specificity in females.

Prefrontal Hemodynamics of Physical Activity and Environmental Complexity During Cognitive Work

OBJECTIVE

The aim of this study was to assess performance and cognitive states during cognitive work in the presence of physical work and in natural settings.

BACKGROUND

Authors of previous studies have examined the interaction between cognitive and physical work, finding performance decrements in working memory. Neuroimaging has revealed increases and decreases in prefrontal oxygenated hemoglobin during the interaction of cognitive and physical work. The effect of environment on cognitive-physical dual tasking has not been previously considered.

METHOD

Thirteen participants were monitored with wireless functional near-infrared spectroscopy (fNIRS) as they performed an auditory 1-back task while sitting, walking indoors, and walking outdoors.

RESULTS

Relative to sitting and walking indoors, auditory working memory performance declined when participants were walking outdoors. Sitting during the auditory 1-back task increased oxygenated hemoglobin and decreased deoxygenated hemoglobin in bilateral prefrontal cortex. Walking reduced the total hemoglobin available to bilateral prefrontal cortex. An increase in environmental complexity reduced oxygenated hemoglobin and increased deoxygenated hemoglobin in bilateral prefrontal cortex.

CONCLUSION

Wireless fNIRS is capable of monitoring cognitive states in naturalistic environments. Selective attention and physical work compete with executive processing. During executive processing loading of selective attention and physical work results in deactivation of bilateral prefrontal cortex and degraded working memory performance, indicating that physical work and concomitant selective attention may supersede executive processing in the distribution of mental resources.

APPLICATION

This research informs decision-making procedures in work where working memory, physical activity, and attention interact. Where working memory is paramount, precautions should be taken to eliminate competition from physical work and selective attention.

Detection and classification of three-class initial dips from prefrontal cortex

In this paper, the use of initial dips using functional near-infrared spectroscopy (fNIRS) for brain-computer interface (BCI) is investigated. Features and window sizes for detecting initial dips are also discussed. Three mental tasks including mental arithmetic, mental counting, and puzzle solving are performed in obtaining fNIRS signals from the prefrontal cortex. Vector-based phase analysis method combined with a threshold circle, as a decision criterion, are used to detect the initial dips. Eight healthy subjects participate in experiment. Linear discriminant analysis is used as a classifier. To classify initial dips, five features (signal mean, peak value, signal slope, skewness, and kurtosis) of oxy-hemoglobin (HbO) and four different window sizes (0~1, 0~1.5, 0~2, and 0~2.5 sec) are examined. It is shown that a combination of signal mean and peak value and a time period of 0~2.5 sec provide the best average classification accuracy of 57.5% for three classes. To further validate the result, three-class classification using the conventional hemodynamic response (HR) is also performed, in which two features (signal mean and signal slope) and 2~7 sec window size have yielded the average classification accuracy of 65.9%. This reveals that fNIRS-based BCI using initial dip detection can reduce the command generation time from 7 sec to 2.5 sec while the classification accuracy is a bit sacrificed from 65.9% to 57.5% for three mental tasks. Further improvement can be made by using deoxy hemoglobin signals in coping with the slow HR problem.

Near-Infrared Spectroscopy based Neurofeedback of Prefrontal Cortex Activity: A Proof-of-Concept Study

Neurofeedback is a promising tool for treatment and rehabilitation of several patient groups. In this proof of principle study, near-infrared spectroscopy (NIRS) based neurofeedback of frontal cortical areas was investigated in healthy adults. Main aims were the assessment of learning, the effects on performance in a working memory (n-back) task and the impact of applied strategies on regulation. 13 healthy participants underwent eight sessions of NIRS based neurofeedback within 2 weeks to learn to voluntarily up-regulate hemodynamic activity in prefrontal areas. An n-back task in pre-/post measurements was used to monitor neurocognitive changes. Mean oxygenated hemoglobin (O2Hb) amplitudes over the course of the sessions as well as during the n-back task were evaluated. 12 out of 13 participants were able to regulate their frontal hemodynamic response via NIRS neurofeedback. However, no systematic learning effects were observed in frontal O2Hb amplitudes over the training course in our healthy sample. We found an impact of applied strategies in only 5 out of 13 subjects. Regarding the n-back task, neurofeedback appeared to induce more focused and specific brain activation compared to pre-training measurement. NIRS based neurofeedback is a feasible and potentially effective method, with an impact on activation patterns in a working memory task. Ceiling effects might explain the lack of a systematic learning pattern in healthy subjects. Clinical studies are needed to show effects in patients exhibiting pathological deviations in prefrontal function.

Left Lateral Prefrontal Activity Reflects a Change of Behavioral Tactics to Cope with a Given Rule: An fNIRS Study

Rules prescribe human behavior and our attempts to choose appropriate behavior under a given rule. Cognitive control, a mechanism to choose and evaluate actions under a rule, is required to determine the appropriate behavior within the limitations of that rule. Consequently, such cognitive control increases mental workload. However, the workload caused by a cognitive task might be different when an additional rule must be considered in choosing the action. The present study was a functional near-infrared spectroscopy (fNIRS) investigation of an experimental task, in which the difficulty of an operation and existence of an additional rule were manipulated to dissociate the influence of that additional rule on cognitive processing. Twenty healthy Japanese volunteers participated. The participants performed an experimental task, in which the player caught one of five colored balls from the upper part of a computer screen by operating a mouse. Four task conditions were prepared to manipulate the task difficulty, which was defined in terms of operational difficulty. In turn, operational difficulty was determined by the width of the playable space and the existence of an additional rule, which reduced the score when a red ball was not caught. The 52-channel fNIRS data were collected from the forehead. Two regions of interest (ROIs) associated with the bilateral lateral prefrontal cortices (LPFCs) were determined, and a three-way repeated-measures analysis of variance (ANOVA) was performed using the task-related signal changes from each ROI. The fNIRS results revealed that bilateral LPFCs showed large signal changes with the increase in mental workload. The ANOVA showed a significant interaction between the existence of an additional rule and the location of the ROIs; that is, the left lateral prefrontal area showed a significant increase in signal intensity when the additional rule existed, and the participant occasionally decided to avoid catching a ball to successfully catch the red-colored ball. Thus, activation of the left LPFC corresponded more closely to the increase in cognitive control underlying the behavioral change made to cope with the additional rule.

Towards psychologically adaptive brain-computer interfaces

OBJECTIVE

Brain-computer interface (BCI) performance is sensitive to short-term changes in psychological states such as fatigue, frustration, and attention. This paper explores the design of a BCI that can adapt to these short-term changes.

APPROACH

Eleven able-bodied individuals participated in a study during which they used a mental task-based EEG-BCI to play a simple maze navigation game while self-reporting their perceived levels of fatigue, frustration, and attention. In an offline analysis, a regression algorithm was trained to predict changes in these states, yielding Pearson correlation coefficients in excess of 0.45 between the self-reported and predicted states. Two means of fusing the resultant mental state predictions with mental task classification were investigated. First, single-trial mental state predictions were used to predict correct classification by the BCI during each trial. Second, an adaptive BCI was designed that retrained a new classifier for each testing sample using only those training samples for which predicted mental state was similar to that predicted for the current testing sample.

MAIN RESULTS

Mental state-based prediction of BCI reliability exceeded chance levels. The adaptive BCI exhibited significant, but practically modest, increases in classification accuracy for five of 11 participants and no significant difference for the remaining six despite a smaller average training set size.

SIGNIFICANCE

Collectively, these findings indicate that adaptation to psychological state may allow the design of more accurate BCIs.

Near-infrared spectroscopy (NIRS)-based eyes-closed brain-computer interface (BCI) using prefrontal cortex activation due to mental arithmetic

We propose a near-infrared spectroscopy (NIRS)-based brain-computer interface (BCI) that can be operated in eyes-closed (EC) state. To evaluate the feasibility of NIRS-based EC BCIs, we compared the performance of an eye-open (EO) BCI paradigm and an EC BCI paradigm with respect to hemodynamic response and classification accuracy. To this end, subjects performed either mental arithmetic or imagined vocalization of the English alphabet as a baseline task with very low cognitive loading. The performances of two linear classifiers were compared; resulting in an advantage of shrinkage linear discriminant analysis (LDA). The classification accuracy of EC paradigm (75.6 ± 7.3%) was observed to be lower than that of EO paradigm (77.0 ± 9.2%), which was statistically insignificant (p = 0.5698). Subjects reported they felt it more comfortable (p = 0.057) and easier (p < 0.05) to perform the EC BCI tasks. The different task difficulty may become a cause of the slightly lower classification accuracy of EC data. From the analysis results, we could confirm the feasibility of NIRS-based EC BCIs, which can be a BCI option that may ultimately be of use for patients who cannot keep their eyes open consistently.

Comparable Cerebral Oxygenation Patterns in Younger and Older Adults during Dual-Task Walking with Increasing Load

The neuroimaging literature on dual-task gait clearly demonstrates increased prefrontal cortex (PFC) involvement when performing a cognitive task while walking. However, findings from direct comparisons of the cerebral oxygenation patterns of younger (YA) and older (OA) adults during dual-task walking are mixed and it is unclear how YA and OA respond to increasing cognitive load (difficulty) while walking. This functional near infra-red (fNIRS) study examined cerebral oxygenation of YA and OA during self-paced dual-task treadmill walking at two different levels of cognitive load (auditory n-back). Changes in accuracy (%) as well as oxygenated (HbO) and deoxygenated (HbR) hemoglobin were examined. For the HbO and HbR measures, eight regions of interest (ROIs) were assessed: the anterior and posterior dorsolateral and ventrolateral PFC (aDLPFC, pDLPFC, aVLPFC, pVLPFC) in each hemisphere. Nineteen YA (M = 21.83 years) and 14 OA (M = 66.85 years) walked at a self-selected pace while performing auditory 1-back and 2-back tasks. Walking alone (single motor: SM) and performing the cognitive tasks alone (single cognitive: SC) were compared to dual-task walking (DT = SM + SC). In the behavioural data, participants were more accurate in the lowest level of load (1-back) compared to the highest (2-back; p < 0.001). YA were more accurate than OA overall (p = 0.009), and particularly in the 2-back task (p = 0.048). In the fNIRS data, both younger and older adults had task effects (SM < DT) in specific ROIs for ΔHbO (three YA, one OA) and ΔHbR (seven YA, eight OA). After controlling for walk speed differences, direct comparisons between YA and OA did not reveal significant age differences, but did reveal a difficulty effect in HbO in the left aDLPFC (p = 0.028) and significant task effects (SM < DT) in HbR for six of the eight ROIs. Findings suggest that YA and OA respond similarly to manipulations of cognitive load when walking on a treadmill at a self-selected pace.

Neural and psychophysiological correlates of human performance under stress and high mental workload

In our anxiogenic and stressful world, the maintenance of an optimal cognitive performance is a constant challenge. It is particularly true in complex working environments (e.g. flight deck, air traffic control tower), where individuals have sometimes to cope with a high mental workload and stressful situations. Several models (i.e. processing efficiency theory, cognitive-energetical framework) have attempted to provide a conceptual basis on how human performance is modulated by high workload and stress/anxiety. These models predict that stress can reduce human cognitive efficiency, even in the absence of a visible impact on the task performance. Performance may be protected under stress thanks to compensatory effort, but only at the expense of a cognitive cost. Yet, the psychophysiological cost of this regulation remains unclear. We designed two experiments involving pupil diameter, cardiovascular and prefrontal oxygenation measurements. Participants performed the Toulouse N-back Task that intensively engaged both working memory and mental calculation processes under the threat (or not) of unpredictable aversive sounds. The results revealed that higher task difficulty (higher n level) degraded the performance and induced an increased tonic pupil diameter, heart rate and activity in the lateral prefrontal cortex, and a decreased phasic pupil response and heart rate variability. Importantly, the condition of stress did not impact the performance, but at the expense of a psychophysiological cost as demonstrated by lower phasic pupil response, and greater heart rate and prefrontal activity. Prefrontal cortex seems to be a central region for mitigating the influence of stress because it subserves crucial functions (e.g. inhibition, working memory) that can promote the engagement of coping strategies. Overall, findings confirmed the psychophysiological cost of both mental effort and stress. Stress likely triggered increased motivation and the recruitment of additional cognitive resources that minimize its aversive effects on task performance (effectiveness), but these compensatory efforts consumed resources that caused a loss of cognitive efficiency (ratio between performance effectiveness and mental effort).

Cortical Signal Analysis and Advances in Functional Near-Infrared Spectroscopy Signal: A Review

Functional near-infrared spectroscopy (fNIRS) is a non-invasive neuroimaging modality that measures the concentration changes of oxy-hemoglobin (HbO) and de-oxy hemoglobin (HbR) at the same time. It is an emerging cortical imaging modality with a good temporal resolution that is acceptable for brain-computer interface applications. Researchers have developed several methods in last two decades to extract the neuronal activation related waveform from the observed fNIRS time series. But still there is no standard method for analysis of fNIRS data. This article presents a brief review of existing methodologies to model and analyze the activation signal. The purpose of this review article is to give a general overview of variety of existing methodologies to extract useful information from measured fNIRS data including pre-processing steps, effects of differential path length factor (DPF), variations and attributes of hemodynamic response function (HRF), extraction of evoked response, removal of physiological noises, instrumentation, and environmental noises and resting/activation state functional connectivity. Finally, the challenges in the analysis of fNIRS signal are summarized.

Prefrontal Cortex Activation and Young Driver Behaviour: A fNIRS Study

Road traffic accidents consistently show a significant over-representation for young, novice and particularly male drivers. This research examines the prefrontal cortex activation of young drivers and the changes in activation associated with manipulations of mental workload and inhibitory control. It also considers the explanation that a lack of prefrontal cortex maturation is a contributing factor to the higher accident risk in this young driver population. The prefrontal cortex is associated with a number of factors including mental workload and inhibitory control, both of which are also related to road traffic accidents. This experiment used functional near infrared spectroscopy to measure prefrontal cortex activity during five simulated driving tasks: one following task and four overtaking tasks at varying traffic densities which aimed to dissociate workload and inhibitory control. Age, experience and gender were controlled for throughout the experiment. The results showed that younger drivers had reduced prefrontal cortex activity compared to older drivers. When both mental workload and inhibitory control increased prefrontal cortex activity also increased, however when inhibitory control alone increased there were no changes in activity. Along with an increase in activity during overtaking manoeuvres, these results suggest that prefrontal cortex activation is more indicative of workload in the current task. There were no differences in the number of overtakes completed by younger and older drivers but males overtook significantly more than females. We conclude that prefrontal cortex activity is associated with the mental workload required for overtaking. We additionally suggest that the reduced activation in younger drivers may be related to a lack of prefrontal maturation which could contribute to the increased crash risk seen in this population.

Walking while Performing Working Memory Tasks Changes the Prefrontal Cortex Hemodynamic Activations and Gait Kinematics

BACKGROUND

Increasing evidence suggests that walking while performing a concurrent task negatively influences gait performance. However, it remains unclear how higher-level cognitive processes and coordination of limb movements are altered in challenging walking environments. This study investigated the influence of cognitive task complexity and walking road condition on the neutral correlates of executive function and postural control in dual-task walking.

METHODS

Twenty-four healthy young adults completed a series of overground walks with three walking road conditions (wide, narrow, with obstacles) with and without the concurrent n-back working memory tasks of two complexity levels (1-back and 3-back). Prefrontal brain activation was assessed by functional near-infrared spectroscopy. A three-dimensional motion analysis system was used simultaneously to measure gait performance and lower-extremity kinematics. Repeated measures analysis of variance were performed to examine the differences between the conditions.

RESULTS

In comparison with standing still, participants showed lower n-back task accuracy while walking, with the worst performance from the road with obstacles. Spatiotemporal gait parameters, lower-extremity joint movements, and the relative changes in oxygenated hemoglobin (HbO) concentration levels were all significantly different across the task complexity and walking path conditions. While dual-tasking participants were found to flex their hips and knees less, leading to a slower gait speed, longer stride time, shorter step length, and greater gait variability than during normal walking. For narrow-road walking, smaller ankle dorsiflexion and larger hip flexion were observed, along with a reduced gait speed. Obstacle negotiation was mainly characterized by increased gait variability than other conditions. HbO levels appeared to be lower during dual-task walking than normal walking. Compared to wide and obstacle conditions, walking on the narrow road was found to elicit a smaller decrement in HbO levels.

CONCLUSION

The current study provided direct evidence that, in young adults, neural correlates of executive function and dynamic postural control tend to be altered in response to the cognitive load imposed by the walking environment and the concurrent task during ambulation. A shift of brain activation patterns between functionally connected networks may occur when facing challenging cognitive-motor interaction.

Into the Wild: Neuroergonomic Differentiation of Hand-Held and Augmented Reality Wearable Displays during Outdoor Navigation with Functional Near Infrared Spectroscopy

Highly mobile computing devices promise to improve quality of life, productivity, and performance. Increased situation awareness and reduced mental workload are two potential means by which this can be accomplished. However, it is difficult to measure these concepts in the “wild”. We employed ultra-portable battery operated and wireless functional near infrared spectroscopy (fNIRS) to non-invasively measure hemodynamic changes in the brain’s Prefrontal cortex (PFC). Measurements were taken during navigation of a college campus with either a hand-held display, or an Augmented reality wearable display (ARWD). Hemodynamic measures were also paired with secondary tasks of visual perception and auditory working memory to provide behavioral assessment of situation awareness and mental workload. Navigating with an augmented reality wearable display produced the least workload during the auditory working memory task, and a trend for improved situation awareness in our measures of prefrontal hemodynamics. The hemodynamics associated with errors were also different between the two devices. Errors with an augmented reality wearable display were associated with increased prefrontal activity and the opposite was observed for the hand-held display. This suggests that the cognitive mechanisms underlying errors between the two devices differ. These findings show fNIRS is a valuable tool for assessing new technology in ecologically valid settings and that ARWDs offer benefits with regards to mental workload while navigating, and potentially superior situation awareness with improved display design.

Transcranial Direct Current Stimulation Modulates Neuronal Activity and Learning in Pilot Training

Skill acquisition requires distributed learning both within (online) and across (offline) days to consolidate experiences into newly learned abilities. In particular, piloting an aircraft requires skills developed from extensive training and practice. Here, we tested the hypothesis that transcranial direct current stimulation (tDCS) can modulate neuronal function to improve skill learning and performance during flight simulator training of aircraft landing procedures. Thirty-two right-handed participants consented to participate in four consecutive daily sessions of flight simulation training and received sham or anodal high-definition-tDCS to the right dorsolateral prefrontal cortex (DLPFC) or left motor cortex (M1) in a randomized, double-blind experiment. Continuous electroencephalography (EEG) and functional near infrared spectroscopy (fNIRS) were collected during flight simulation, n-back working memory, and resting-state assessments. tDCS of the right DLPFC increased midline-frontal theta-band activity in flight and n-back working memory training, confirming tDCS-related modulation of brain processes involved in executive function. This modulation corresponded to a significantly different online and offline learning rates for working memory accuracy and decreased inter-subject behavioral variability in flight and n-back tasks in the DLPFC stimulation group. Additionally, tDCS of left M1 increased parietal alpha power during flight tasks and tDCS to the right DLPFC increased midline frontal theta-band power during n-back and flight tasks. These results demonstrate a modulation of group variance in skill acquisition through an increasing in learned skill consistency in cognitive and real-world tasks with tDCS. Further, tDCS performance improvements corresponded to changes in electrophysiological and blood-oxygenation activity of the DLPFC and motor cortices, providing a stronger link between modulated neuronal function and behavior.

Decoding four different sound-categories in the auditory cortex using functional near-infrared spectroscopy

The ability of the auditory cortex in the brain to distinguish different sounds is important in daily life. This study investigated whether activations in the auditory cortex caused by different sounds can be distinguished using functional near-infrared spectroscopy (fNIRS). The hemodynamic responses (HRs) in both hemispheres using fNIRS were measured in 18 subjects while exposing them to four sound categories (English-speech, non-English-speech, annoying sounds, and nature sounds). As features for classifying the different signals, the mean, slope, and skewness of the oxy-hemoglobin (HbO) signal were used. With regard to the language-related stimuli, the HRs evoked by understandable speech (English) were observed in a broader brain region than were those evoked by non-English speech. Also, the magnitudes of the HbO signals evoked by English-speech were higher than those of non-English speech. The ratio of the peak values of non-English and English speech was 72.5%. Also, the brain region evoked by annoying sounds was wider than that by nature sounds. However, the signal strength for nature sounds was stronger than that for annoying sounds. Finally, for brain-computer interface (BCI) purposes, the linear discriminant analysis (LDA) and support vector machine (SVM) classifiers were applied to the four sound categories. The overall classification performance for the left hemisphere was higher than that for the right hemisphere. Therefore, for decoding of auditory commands, the left hemisphere is recommended. Also, in two-class classification, the annoying vs. nature sounds comparison provides a higher classification accuracy than the English vs. non-English speech comparison. Finally, LDA performs better than SVM.

High User Control in Game Design Elements Increases Compliance and In-game Performance in a Memory Training Game

Computer games are increasingly being used for training cognitive functions like working memory and attention among the growing population of older adults. While cognitive training games often include elements like difficulty adaptation, rewards, and visual themes to make the games more enjoyable and effective, the effect of different degrees of afforded user control in manipulating these elements has not been systematically studied. To address this issue, two distinct implementations of the three aforementioned game elements were tested among healthy older adults (N = 21, 69.9 ± 6.4 years old) playing a game-like version of the n-back task on a tablet at home for 3 weeks. Two modes were considered, differentiated by the afforded degree of user control of the three elements: user control of difficulty vs. automatic difficulty adaptation, difficulty-dependent rewards vs. automatic feedback messages, and user choice of visual theme vs. no choice. The two modes ("USER-CONTROL" and "AUTO") were compared for frequency of play, duration of play, and in-game performance. Participants were free to play the game whenever and for however long they wished. Participants in USER-CONTROL exhibited significantly higher frequency of playing, total play duration, and in-game performance than participants in AUTO. The results of the present study demonstrate the efficacy of providing user control in the three game elements, while validating a home-based study design in which participants were not bound by any training regimen, and could play the game whenever they wished. The results have implications for designing cognitive training games that elicit higher compliance and better in-game performance, with an emphasis on home-based training.

Toward a Wireless Open Source Instrument: Functional Near-infrared Spectroscopy in Mobile Neuroergonomics and BCI Applications

Brain-Computer Interfaces (BCIs) and neuroergonomics research have high requirements regarding robustness and mobility. Additionally, fast applicability and customization are desired. Functional Near-Infrared Spectroscopy (fNIRS) is an increasingly established technology with a potential to satisfy these conditions. EEG acquisition technology, currently one of the main modalities used for mobile brain activity assessment, is widely spread and open for access and thus easily customizable. fNIRS technology on the other hand has either to be bought as a predefined commercial solution or developed from scratch using published literature. To help reducing time and effort of future custom designs for research purposes, we present our approach toward an open source multichannel stand-alone fNIRS instrument for mobile NIRS-based neuroimaging, neuroergonomics and BCI/BMI applications. The instrument is low-cost, miniaturized, wireless and modular and openly documented on www.opennirs.org. It provides features such as scalable channel number, configurable regulated light intensities, programmable gain and lock-in amplification. In this paper, the system concept, hardware, software and mechanical implementation of the lightweight stand-alone instrument are presented and the evaluation and verification results of the instrument's hardware and physiological fNIRS functionality are described. Its capability to measure brain activity is demonstrated by qualitative signal assessments and a quantitative mental arithmetic based BCI study with 12 subjects.

Towards a ternary NIRS-BCI: single-trial classification of verbal fluency task, Stroop task and unconstrained rest

OBJECTIVE

The majority of near-infrared spectroscopy (NIRS) brain-computer interface (BCI) studies have investigated binary classification problems. Limited work has considered differentiation of more than two mental states, or multi-class differentiation of higher-level cognitive tasks using measurements outside of the anterior prefrontal cortex. Improvements in accuracies are needed to deliver effective communication with a multi-class NIRS system. We investigated the feasibility of a ternary NIRS-BCI that supports mental states corresponding to verbal fluency task (VFT) performance, Stroop task performance, and unconstrained rest using prefrontal and parietal measurements.

APPROACH

Prefrontal and parietal NIRS signals were acquired from 11 able-bodied adults during rest and performance of the VFT or Stroop task. Classification was performed offline using bagging with a linear discriminant base classifier trained on a 10 dimensional feature set.

MAIN RESULTS

VFT, Stroop task and rest were classified at an average accuracy of 71.7% ± 7.9%. The ternary classification system provided a statistically significant improvement in information transfer rate relative to a binary system controlled by either mental task (0.87 ± 0.35 bits/min versus 0.73 ± 0.24 bits/min).

SIGNIFICANCE

These results suggest that effective communication can be achieved with a ternary NIRS-BCI that supports VFT, Stroop task and rest via measurements from the frontal and parietal cortices. Further development of such a system is warranted. Accurate ternary classification can enhance communication rates offered by NIRS-BCIs, improving the practicality of this technology.

Correlates of Near-Infrared Spectroscopy Brain-Computer Interface Accuracy in a Multi-Class Personalization Framework

Brain–computer interfaces (BCIs) provide individuals with a means of interacting with a computer using only neural activity. To date, the majority of near-infrared spectroscopy (NIRS) BCIs have used prescribed tasks to achieve binary control. The goals of this study were to evaluate the possibility of using a personalized approach to establish control of a two-, three-, four-, and five-class NIRS–BCI, and to explore how various user characteristics correlate to accuracy. Ten able-bodied participants were recruited for five data collection sessions. Participants performed six mental tasks and a personalized approach was used to select each individual’s best discriminating subset of tasks. The average offline cross-validation accuracies achieved were 78, 61, 47, and 37% for the two-, three-, four-, and five-class problems, respectively. Most notably, all participants exceeded an accuracy of 70% for the two-class problem, and two participants exceeded an accuracy of 70% for the three-class problem. Additionally, accuracy was found to be strongly positively correlated (Pearson’s) with perceived ease of session (ρ = 0.653), ease of concentration (ρ = 0.634), and enjoyment (ρ = 0.550), but strongly negatively correlated with verbal IQ (ρ = −0.749).

Brain-machine interfaces for assistive smart homes: A feasibility study with wearable near-infrared spectroscopy

Smart houses for elderly or physically challenged people need a method to understand residents' intentions during their daily-living behaviors. To explore a new possibility, we here developed a novel brain-machine interface (BMI) system integrated with an experimental smart house, based on a prototype of a wearable near-infrared spectroscopy (NIRS) device, and verified the system in a specific task of controlling of the house's equipments with BMI. We recorded NIRS signals of three participants during typical daily-living actions (DLAs), and classified them by linear support vector machine. In our off-line analysis, four DLAs were classified at about 70% mean accuracy, significantly above the chance level of 25%, in every participant. In an online demonstration in the real smart house, one participant successfully controlled three target appliances by BMI at 81.3% accuracy. Thus we successfully demonstrated the feasibility of using NIRS-BMI in real smart houses, which will possibly enhance new assistive smart-home technologies.

Investigating deep learning for fNIRS based BCI

Functional Near infrared Spectroscopy (fNIRS) is a relatively young modality for measuring brain activity which has recently shown promising results for building Brain Computer Interfaces (BCI). Due to its infancy, there are still no standard approaches for meaningful features and classifiers for single trial analysis of fNIRS. Most studies are limited to established classifiers from EEG-based BCIs and very simple features. The feasibility of more complex and powerful classification approaches like Deep Neural Networks has, to the best of our knowledge, not been investigated for fNIRS based BCI. These networks have recently become increasingly popular, as they outperformed conventional machine learning methods for a variety of tasks, due in part to advances in training methods for neural networks. In this paper, we show how Deep Neural Networks can be used to classify brain activation patterns measured by fNIRS and compare them with previously used methods.

Single-trial classification of near-infrared spectroscopy signals arising from multiple cortical regions

Near-infrared spectroscopy (NIRS) brain-computer interface (BCI) studies have primarily made use of measurements taken from a single cortical area. In particular, the anterior prefrontal cortex has been the key area used for detecting higher-level cognitive task performance. However, mental task execution typically requires coordination between several, spatially-distributed brain regions. We investigated the value of expanding the area of interrogation to include NIRS measurements from both the prefrontal and parietal cortices to decode mental states. Hemodynamic activity was monitored at 46 locations over the prefrontal and parietal cortices using a continuous-wave near-infrared spectrometer while 11 able-bodied adults rested or performed either the verbal fluency task (VFT) or Stroop task. Offline classification was performed for the three possible binary problems using 25 iterations of bagging with a linear discriminant base classifier. Classifiers were trained on a 10 dimensional feature set. When all 46 measurement locations were considered for classification, average accuracies of 80.4±7.0%, 82.4±7.6%, and 82.8±5.9% in differentiating VFT vs rest, Stroop vs rest and VFT vs Stroop, respectively, were obtained. Relative to using measurements from the anterior PFC alone, an overall average improvement of 11.3% was achieved. Utilizing NIRS measurements from the prefrontal and parietal cortices can be of value in classifying mental states involving working memory and attention. NIRS-BCI accuracies may be improved by incorporating measurements from several, distinct cortical regions, rather than a single area alone. Further development of an NIRS-BCI supporting combinations of VFT, Stroop task and rest states is also warranted.

Wearable functional near infrared spectroscopy (fNIRS) and transcranial direct current stimulation (tDCS): expanding vistas for neurocognitive augmentation

Contemporary studies with transcranial direct current stimulation (tDCS) provide a growing base of evidence for enhancing cognition through the non-invasive delivery of weak electric currents to the brain. The main effect of tDCS is to modulate cortical excitability depending on the polarity of the applied current. However, the underlying mechanism of neuromodulation is not well understood. A new generation of functional near infrared spectroscopy (fNIRS) systems is described that are miniaturized, portable, and include wearable sensors. These developments provide an opportunity to couple fNIRS with tDCS, consistent with a neuroergonomics approach for joint neuroimaging and neurostimulation investigations of cognition in complex tasks and in naturalistic conditions. The effects of tDCS on complex task performance and the use of fNIRS for monitoring cognitive workload during task performance are described. Also explained is how fNIRS + tDCS can be used simultaneously for assessing spatial working memory. Mobile optical brain imaging is a promising neuroimaging tool that has the potential to complement tDCS for realistic applications in natural settings.

Single-trial classification of NIRS data from prefrontal cortex during working memory tasks

This study presents single-trial classification performance on high density Near Infrared Spectroscopy (NIRS) data collected from the prefrontal cortex of 11 healthy subjects while performing working memory tasks and idle condition. The NIRS data collected comprised a total of 40 trials of n-back tasks for 2 difficulty levels: n=1 for easy and n=3 for hard. The single-trial classification was performed on features extracted using common average reference spatial filtering and single-trial baseline reference. The single-trial classification was performed using 5×5-fold cross-validations on the NIRS data collected by using mutual information-based feature selection and the support vector machine classifier. The results yielded average accuracies of 72.7%, 68.0% and 84.0% in classifying hard versus easy tasks, easy versus idle tasks and hard versus idle tasks respectively. Hence the results demonstrated a potential feasibility of using high density NIRS-based BCI for assessing working memory load.

Hybrid fNIRS-EEG based classification of auditory and visual perception processes

For multimodal Human-Computer Interaction (HCI), it is very useful to identify the modalities on which the user is currently processing information. This would enable a system to select complementary output modalities to reduce the user's workload. In this paper, we develop a hybrid Brain-Computer Interface (BCI) which uses Electroencephalography (EEG) and functional Near Infrared Spectroscopy (fNIRS) to discriminate and detect visual and auditory stimulus processing. We describe the experimental setup we used for collection of our data corpus with 12 subjects. On this data, we performed cross-validation evaluation, of which we report accuracy for different classification conditions. The results show that the subject-dependent systems achieved a classification accuracy of 97.8% for discriminating visual and auditory perception processes from each other and a classification accuracy of up to 94.8% for detecting modality-specific processes independently of other cognitive activity. The same classification conditions could also be discriminated in a subject-independent fashion with accuracy of up to 94.6 and 86.7%, respectively. We also look at the contributions of the two signal types and show that the fusion of classifiers using different features significantly increases accuracy.

What we can and cannot (yet) do with functional near infrared spectroscopy

Functional near infrared spectroscopy (NIRS) is a relatively new technique complimentary to EEG for the development of brain-computer interfaces (BCIs). NIRS-based systems for detecting various cognitive and affective states such as mental and emotional stress have already been demonstrated in a range of adaptive human–computer interaction (HCI) applications. However, before NIRS-BCIs can be used reliably in realistic HCI settings, substantial challenges oncerning signal processing and modeling must be addressed. Although many of those challenges have been identified previously, the solutions to overcome them remain scant. In this paper, we first review what can be currently done with NIRS, specifically, NIRS-based approaches to measuring cognitive and affective user states as well as demonstrations of passive NIRS-BCIs. We then discuss some of the primary challenges these systems would face if deployed in more realistic settings, including detection latencies and motion artifacts. Lastly, we investigate the effects of some of these challenges on signal reliability via a quantitative comparison of three NIRS models. The hope is that this paper will actively engage researchers to acilitate the advancement of NIRS as a more robust and useful tool to the BCI community.