Single-trial lie detection using a combined fNIRS-polygraph system

Artificial neural network models: implementation of functional near-infrared spectroscopy-based spontaneous lie detection in an interactive scenario

Deception is an inevitable occurrence in daily life. Various methods have been used to understand the mechanisms underlying brain deception. Moreover, numerous efforts have been undertaken to detect deception and truth-telling. Functional near-infrared spectroscopy (fNIRS) has great potential for neurological applications compared with other state-of-the-art methods. Therefore, an fNIRS-based spontaneous lie detection model was used in the present study. We interviewed 10 healthy subjects to identify deception using the fNIRS system. A card game frequently referred to as a bluff or cheat was introduced. This game was selected because its rules are ideal for testing our hypotheses. The optical probe of the fNIRS was placed on the subject's forehead, and we acquired optical density signals, which were then converted into oxy-hemoglobin and deoxy-hemoglobin signals using the Modified Beer-Lambert law. The oxy-hemoglobin signal was preprocessed to eliminate noise. In this study, we proposed three artificial neural networks inspired by deep learning models, including AlexNet, ResNet, and GoogleNet, to classify deception and truth-telling. The proposed models achieved accuracies of 88.5%, 88.0%, and 90.0%, respectively. These proposed models were compared with other classification models, including k-nearest neighbor, linear support vector machines (SVM), quadratic SVM, cubic SVM, simple decision trees, and complex decision trees. These comparisons showed that the proposed models performed better than the other state-of-the-art methods.

Electrodermal signal analysis using continuous wavelet transform as a tool for quantification of sweat gland activity in diabetic kidney disease

Sympathetic innervation of the sweat gland (SG) manifests itself electrically as electrodermal activity (EDA), which can be utilized to measure sudomotor function. Since SG exhibits similarities in structure and function with kidneys, quantification of SG activity is attempted through EDA signals. A methodology is developed with electrical stimulation, sampling frequency and signal processing algorithm. One hundred twenty volunteers participated in this study belonging to controls, diabetes, diabetic nephropathy, and diabetic neuropathy. The magnitude and time duration of stimuli is arrived by trial and error in such a way it does not influence controls but triggers SG activity in other Groups. This methodology leads to a distinct EDA signal pattern with changes in frequency and amplitude. The continuous wavelet transform depicts a scalogram to retrieve this information. Further, to distinguish between Groups, time average spectrums are plotted and mean relative energy (MRE) is computed. Results demonstrate high energy value in controls, and it gradually decreases in other Groups indicating a decline in SG activity on diabetes prognosis. The correlation for the acquired results was determined to be 0.99 when compared to the standard lab procedure. Furthermore, Cohen's d value, which is less than 0.25 for all Groups indicating the minimal effect size. Hence the obtained result is validated and statistically analyzed for individual variations. Thus this has the potential to get transformed into a device and could prevent diabetic kidney disease.

Novel Approaches and Cognitive Neuroscience Perspectives on False Memory and Deception

The DRM (Deese-Roediger-McDermott) paradigm produces robust false memories of non-presented critical words. After studying a thematic word list (e.g., bed, rest, and pillow) participants falsely remember the critical item "sleep." We report two false memory experiments. Study One introduces a novel use of the lexical decision task (LDT) to prime critical words. Participants see two letter-strings and make timed responses indicating whether they are both words. The word pairs Night-Bed and Dream-Thweeb both prime "sleep" but only one pair contains two words. Our primary purpose is to introduce this new methodology via two pilot experiments. The results, considered preliminary, are promising as they indicate that participants were as likely to recognize critical words (false memories) and presented words (true memories) just as when studying thematic lists. Study Two actually employs the standard DRM lists so that semantic priming is in play there as well. The second study, however, uses functional near-infrared spectroscopy (fNIRS) to measure activity in the prefrontal cortex during a DRM task which includes a deception phase where participants intentionally lie about critical lures. False and true memories occurred at high levels and activated many of the same brain regions but, compared to true memories, cortical activity was higher for false memories and lies. Accuracy findings are accompanied by confidence and reaction time results. Both investigations suggest that it is difficult to distinguish accurate from inaccurate memories. We explain results in terms of activation-monitoring theory and Fuzzy Trace Theory. We provide real world implications and suggest extending the present research to varying age groups and special populations. A nagging question has not been satisfactorily answered: Could neural pathways exist that signal the presence of false memories and lies? Answering this question will require imaging experiments that focus on regions of distinction such as the anterior prefrontal cortex.

Detecting concealed information using functional near-infrared spectroscopy (fNIRS) combined with skin conductance, heart rate, and behavioral measures

In this study, brain imaging data from functional near-infrared spectroscopy (fNIRS) associated with skin conductance response (SCR), heart rate (HR), and reaction time (RT) were combined to determine if the combination of these indicators could improve the efficiency of deception detection in concealed information test (CIT). During the CIT, participants were presented with a series of names and cities that served as target, probe, or irrelevant stimuli. In the guilty group, the probe stimuli were the participants' own names and hometown cities, and they were asked to deny this information. Our results revealed that probe items were associated with longer RT, larger SCR, slower HR, and higher oxyhemoglobin (HbO) concentration changes in the inferior prefrontal gyrus (IFG), middle frontal gyrus (MFG), and the superior frontal gyrus (SFG) compared with irrelevant items for participants in the guilty group but not in the innocent group. Furthermore, our results suggested that the combination of RT, SCR, HR, and fNIRS indicators could improve the deception detection efficiency to a very high area under the ROC curve (0.94) compared with any of the single indicators (0.74-0.89). The improved deception detection efficiency might be attributed to the reduction of random error and the diversiform underlying the psychophysiological mechanisms reflected by each indicator. These findings demonstrate a feasible way to improve the deception detection efficiency by using combined multiple indicators.

Machine Learning-Based Lie Detector Applied to a Novel Annotated Game Dataset

Lie detection is considered a concern for everyone in their day-to-day life, given its impact on human interactions. Thus, people normally pay attention to both what their interlocutors are saying and to their visual appearance, including the face, to find any signs that indicate whether or not the person is telling the truth. While automatic lie detection may help us to understand these lying characteristics, current systems are still fairly limited, partly due to lack of adequate datasets to evaluate their performance in realistic scenarios. In this work, we collect an annotated dataset of facial images, comprising both 2D and 3D information of several participants during a card game that encourages players to lie. Using our collected dataset, we evaluate several types of machine learning-based lie detectors in terms of their generalization, in person-specific and cross-application experiments. We first extract both handcrafted and deep learning-based features as relevant visual inputs, then pass them into multiple types of classifier to predict respective lie/non-lie labels. Subsequently, we use several metrics to judge the models’ accuracy based on the models predictions and ground truth. In our experiment, we show that models based on deep learning achieve the highest accuracy, reaching up to 57% for the generalization task and 63% when applied to detect the lie to a single participant. We further highlight the limitation of the deep learning-based lie detector when dealing with cross-application lie detection tasks. Finally, this analysis along the proposed datasets would potentially be useful not only from the perspective of computational systems perspective (e.g., improving current automatic lie prediction accuracy), but also for other relevant application fields, such as health practitioners in general medical counselings, education in academic settings or finance in the banking sector, where close inspections and understandings of the actual intentions of individuals can be very important.

The role of anterior prefrontal cortex (area 10) in face-to-face deception measured with fNIRS

Anterior prefrontal cortex (PFC, Brodmann area 10) activations are often, but not always, found in neuroimaging studies investigating deception, and the precise role of this area remains unclear. To explore the role of the PFC in face-to-face deception, we invited pairs of participants to play a card game involving lying and lie detection while we used functional near infrared spectroscopy (fNIRS) to record brain activity in the PFC. Participants could win points for successfully lying about the value of their cards or for detecting lies. We contrasted patterns of brain activation when the participants either told the truth or lied, when they were either forced into this or did so voluntarily and when they either succeeded or failed to detect a lie. Activation in the anterior PFC was found in both lie production and detection, unrelated to reward. Analysis of cross-brain activation patterns between participants identified areas of the PFC where the lead player’s brain activity synchronized their partner’s later brain activity. These results suggest that during situations that involve close interpersonal interaction, the anterior PFC supports processing widely involved in deception, possibly relating to the demands of monitoring one’s own and other people’s behaviour.

Decoding Multiple Sound-Categories in the Auditory Cortex by Neural Networks: An fNIRS Study

This study aims to decode the hemodynamic responses (HRs) evoked by multiple sound-categories using functional near-infrared spectroscopy (fNIRS). The six different sounds were given as stimuli (English, non-English, annoying, nature, music, and gunshot). The oxy-hemoglobin (HbO) concentration changes are measured in both hemispheres of the auditory cortex while 18 healthy subjects listen to 10-s blocks of six sound-categories. Long short-term memory (LSTM) networks were used as a classifier. The classification accuracy was 20.38 ± 4.63% with six class classification. Though LSTM networks' performance was a little higher than chance levels, it is noteworthy that we could classify the data subject-wise without feature selections.

Decoding of Walking Imagery and Idle State Using Sparse Representation Based on fNIRS

Objectives

Brain-computer interface (BCI) based on functional near-infrared spectroscopy (fNIRS) is expected to provide an optional active rehabilitation training method for patients with walking dysfunction, which will affect their quality of life seriously. Sparse representation classification (SRC) oxyhemoglobin (HbO) concentration was used to decode walking imagery and idle state to construct fNIRS-BCI based on walking imagery.

Methods

15 subjects were recruited and fNIRS signals were collected during walking imagery and idle state. Firstly, band-pass filtering and baseline drift correction for HbO signal were carried out, and then the mean value, peak value, and root mean square (RMS) of HbO and their combinations were extracted as classification features; SRC was used to identify the extracted features and the result of SRC was compared with those of support vector machine (SVM), K-Nearest Neighbor (KNN), linear discriminant analysis (LDA), and logistic regression (LR).

Results

The experimental results showed that the average classification accuracy for walking imagery and idle state by SRC using three features combination was 91.55±3.30%, which was significantly higher than those of SVM, KNN, LDA, and LR (86.37±4.42%, 85.65±5.01%, 86.43±4.41%, and 76.14±5.32%, respectively), and the classification accuracy of other combined features was higher than that of single feature.

Conclusions

The study showed that introducing SRC into fNIRS-BCI can effectively identify walking imagery and idle state. It also showed that different time windows for feature extraction have an impact on the classification results, and the time window of 2–8 s achieved a better classification accuracy (94.33±2.60%) than other time windows. Significance. The study was expected to provide a new and optional active rehabilitation training method for patients with walking dysfunction. In addition, the experiment was also a rare study based on fNIRS-BCI using SRC to decode walking imagery and idle state.

A Look Into the Power of fNIRS Signals by Using the Welch Power Spectral Estimate for Deception Detection

Neuroimaging technologies have improved our understanding of deception and also exhibit their potential in revealing the origins of its neural mechanism. In this study, a quantitative power analysis method that uses the Welch power spectrum estimation of functional near-infrared spectroscopy (fNIRS) signals was proposed to examine the brain activation difference between the spontaneous deceptive behavior and controlled behavior. The power value produced by the model was applied to quantify the activity energy of brain regions, which can serve as a neuromarker for deception detection. Interestingly, the power analysis results generated from the Welch spectrum estimation method demonstrated that the spontaneous deceptive behavior elicited significantly higher power than that from the controlled behavior in the prefrontal cortex. Meanwhile, the power findings also showed significant difference between the spontaneous deceptive behavior and controlled behavior, indicating that the reward system was only involved in the deception. The proposed power analysis method for processing fNIRS data provides us an additional insight to understand the cognitive mechanism of deception.

LieToMe: An Ensemble Approach for Deception Detection from Facial Cues

Deception detection is a relevant ability in high stakes situations such as police interrogatories or court trials, where the outcome is highly influenced by the interviewed person behavior. With the use of specific devices, e.g. polygraph or magnetic resonance, the subject is aware of being monitored and can change his behavior, thus compromising the interrogation result. For this reason, video analysis-based methods for automatic deception detection are receiving ever increasing interest. In this paper, a deception detection approach based on RGB videos, leveraging both facial features and stacked generalization ensemble, is proposed. First, a face, which is well-known to present several meaningful cues for deception detection, is identified, aligned, and masked to build video signatures. These signatures are constructed starting from five different descriptors, which allow the system to capture both static and dynamic facial characteristics. Then, video signatures are given as input to four base-level algorithms, which are subsequently fused applying the stacked generalization technique, resulting in a more robust meta-level classifier used to predict deception. By exploiting relevant cues via specific features, the proposed system achieves improved performances on a public dataset of famous court trials, with respect to other state-of-the-art methods based on facial features, highlighting the effectiveness of the proposed method.

The Role of Reward System in Dishonest Behavior: A Functional Near-Infrared Spectroscopy Study

Previous studies showed that the cortical reward system plays an important role in deceptive behavior. However, how the reward system activates during the whole course of dishonest behavior and how it affects dishonest decisions remain unclear. The current study investigated these questions. One hundred and two participants were included in the final analysis. They completed two tasks: monetary incentive delay (MID) task and an honesty task. The MID task served as the localizer task and the honesty task was used to measure participants' deceptive behaviors. Participants' spontaneous responses in the honesty task were categorized into three conditions: Correct-Truth condition (tell the truth after guessing correctly), Incorrect-Truth condition (tell the truth after guessing incorrectly), and Incorrect-Lie condition (tell lies after guessing incorrectly). To reduce contamination from neighboring functional regions as well as to increase sensitivity to small effects (Powell et al., Devel Sci 21:e12595, 2018), we adopted the individual functional channel of interest (fCOI) approach to analyze the data. Specially, we identified the channels of interest in the MID task in individual participants and then applied them to the honesty task. The result suggested that the reward system showed different activation patterns during different phases: In the pre-decision phase, the reward system was activated with the winning of the reward. During the decision and feedback phase, the reward system was activated when people made the decisions to be dishonest and when they evaluated the outcome of their decisions. Furthermore, the result showed that neural activity of the reward system toward the outcome of their decision was related to subsequent dishonest behaviors. Thus, the present study confirmed the important role of the reward system in deception. These results can also shed light on how one could use neuroimaging techniques to perform lie-detection.

Reinforcement-Sensitive Personality Traits Associated With Passion in Heterosexual Intimate Relationships: An fNIRS Investigation

According to the triangular theory of love, passion is an indispensable component of romantic love. Some brain imaging studies have shown that passionate arousal in intimate relationships is associated with the reward circuits in the brain. We hypothesized that the individual reward sensitivity trait is also related to passion in intimate relationships, and two separate studies were conducted in the present research. In the first study, 558 college students who were currently in love were selected as participants. The correlation between intimacy and reinforcement sensitivity in individuals identifying as heterosexual was explored using the Sensitivity to Punishment and Sensitivity to Reward Questionnaire, the Passionate Love Scale, and the Triangular Love Scale. In the second study, participants were 42 college students who were also currently in love. Functional near-infrared spectroscopy (fNIRS) was adopted to explore the neurophysiological interaction between reward sensitivity and emotional arousal induced in participants when presented a photograph of their partner, a friend, or a stranger. The results showed that reward sensitivity was positively correlated with passion, and punishment sensitivity was negatively correlated with intimacy and commitment. Significant interactions between reward sensitivity and photograph type were found, and the triangular part of the inferior frontal gyrus showed a particular relevance to the reward-sensitive personality trait toward partners. Overall, the findings support reinforcement sensitivity theory and suggest that reinforcement-sensitive personality traits (personality traits of reward and punishment sensitivity) are associated with all three components of love, with only reward sensitivity being related to passion.

Random Subspace Ensemble Learning for Functional Near-Infrared Spectroscopy Brain-Computer Interfaces

The feasibility of the random subspace ensemble learning method was explored to improve the performance of functional near-infrared spectroscopy-based brain-computer interfaces (fNIRS-BCIs). Feature vectors have been constructed using the temporal characteristics of concentration changes in fNIRS chromophores such as mean, slope, and variance to implement fNIRS-BCIs systems. The mean and slope, which are the most popular features in fNIRS-BCIs, were adopted. Linear support vector machine and linear discriminant analysis were employed, respectively, as a single strong learner and multiple weak learners. All features in every channel and available time window were employed to train the strong learner, and the feature subsets were selected at random to train multiple weak learners. It was determined that random subspace ensemble learning is beneficial to enhance the performance of fNIRS-BCIs.

Using Polygraph to Detect Passengers Carrying Illegal Items

The present study examined the effectiveness of a Modified-Comparison Questions Technique, used in conjunction with the polygraph, to differentiate between common travelers, drug traffickers, and terrorists at transportation hubs. Two experiments were conducted using a mock crime paradigm. In Experiment 1, we randomly assigned 78 participants to either a drug condition, where they packed and lied about illicit drugs in their luggage, or a control condition, where they did not pack or lie about any illegal items. In Experiment 2, we randomly assigned 164 participants to one of the two conditions in Experiment 1 or an additional bomb condition, where they packed and lied about a bomb in their luggage. For both experiments, we assessed participants’ RR interval, heart rate, peak-to-peak amplitude of Galvanic Skin Response (GSR) and all three combined, using Discriminant Analyses to determine the classification accuracy of participants in each condition. In both experiments, we found decelerated heart rates and increased peak-to-peak amplitude of GSR in guilty participants when lying in response to questions regarding their crime. We also found accurate classifications of participants, in both Experiment 1 (drug vs. control: 84.2% vs. 82.5%) and Experiment 2 (drug vs. control: 82:1% vs. 95.1%; bomb vs. control: 93.2% vs. 95.1%; drug vs. bomb: 92.3% vs. 90.9%), above chance level. These findings indicate that Modified-CQT, combined with a polygraph test, is a viable method for investigating suspects of drug trafficking and terrorism at transportation hubs such as train stations and airports.

Adaptive filtering of physiological noises in fNIRS data

The study presents a recursive least-squares estimation method with an exponential forgetting factor for noise removal in functional near-infrared spectroscopy data and extraction of hemodynamic responses (HRs) from the measured data. The HR is modeled as a linear regression form in which the expected HR, the first and second derivatives of the expected HR, a short-separation measurement data, three physiological noises, and the baseline drift are included as components in the regression vector. The proposed method is applied to left-motor-cortex experiments on the right thumb and little finger movements in five healthy male participants. The algorithm is evaluated with respect to its performance improvement in terms of contrast-to-noise ratio in comparison with Kalman filter, low-pass filtering, and independent component method. The experimental results show that the proposed model achieves reductions of 77% and 99% in terms of the number of channels exhibiting higher contrast-to-noise ratios in oxy-hemoglobin and deoxy-hemoglobin, respectively. The approach is robust in obtaining consistent HR data. The proposed method is applied for both offline and online noise removal.

Detecting Concealed Information with Fused Electroencephalography and Functional Near-infrared Spectroscopy

In this study, fused electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) techniques were utilized to examine the relationship between the ERP (event-related potential) component P300 and fNIRS hemodynamic signals for high-accuracy deception detection. During the performance of a modified concealed information test (CIT) task, a series of Chinese names were presented, which served as the target, irrelevant, or the probe stimuli for both the guilty and innocent groups. For participants in the guilty group, the probe stimulus was their individual name, whereas for the innocent group, the probe stimulus was one irrelevant name. In particular, data from concurrent fNIRS and ERP recordings were carefully inspected for participants from the two groups. Interestingly, we discovered that for the guilty group, the probe stimulus elicited significantly higher P300 amplitude at parietal site and also evoked significantly stronger oxyhemoglobin (HbO) concentration changes in the bilateral superior frontal gyrus and bilateral middle frontal gyrus than the irrelevant stimuli. However, this is not the case for the innocent group, in which participants exhibited no significant differences in both ERP and fNIRS measures between the probe and irrelevant stimuli. More importantly, our findings also demonstrated that the combined ERP and fNIRS feature was able to differentiate the guilty and innocent groups with enhanced sensitivity, in which AUC (the area under Receiver Operating Characteristic curve) is 0.94 for deception detection based on the combined indicator, much higher than that based on the ERP component P300 only (0.85) or HbO measure only (0.84).

Feature Extraction and Classification Methods for Hybrid fNIRS-EEG Brain-Computer Interfaces

In this study, a brain-computer interface (BCI) framework for hybrid functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG) for locked-in syndrome (LIS) patients is investigated. Brain tasks, channel selection methods, and feature extraction and classification algorithms available in the literature are reviewed. First, we categorize various types of patients with cognitive and motor impairments to assess the suitability of BCI for each of them. The prefrontal cortex is identified as a suitable brain region for imaging. Second, the brain activity that contributes to the generation of hemodynamic signals is reviewed. Mental arithmetic and word formation tasks are found to be suitable for use with LIS patients. Third, since a specific targeted brain region is needed for BCI, methods for determining the region of interest are reviewed. The combination of a bundled-optode configuration and threshold-integrated vector phase analysis turns out to be a promising solution. Fourth, the usable fNIRS features and EEG features are reviewed. For hybrid BCI, a combination of the signal peak and mean fNIRS signals and the highest band powers of EEG signals is promising. For classification, linear discriminant analysis has been most widely used. However, further research on vector phase analysis as a classifier for multiple commands is desirable. Overall, proper brain region identification and proper selection of features will improve classification accuracy. In conclusion, five future research issues are identified, and a new BCI scheme, including brain therapy for LIS patients and using the framework of hybrid fNIRS-EEG BCI, is provided.

Lie Detection Using fNIRS Monitoring of Inhibition-Related Brain Regions Discriminates Infrequent but not Frequent Liars

Functional near-infrared spectroscopy (fNIRS) was used to test whether monitoring inhibition-related brain regions is a feasible method for detecting both infrequent liars and frequent liars. Thirty-two participants were divided into two groups: the deceptive group (liars) and the non-deceptive group (ND group, innocents). All the participants were required to undergo a simulated interrogation by a computer. The participants from the deceptive group were instructed to tell a mix of lies and truths and those of the ND group were instructed always to tell the truth. Based on the number of deceptions, the participants of the deceptive group were further divided into a infrequently deceptive group (IFD group, infrequent liars) and a frequently deceptive group (FD group, frequent liars). The infrequent liars exhibited greater neural activities than the frequent liars and the innocents in the left middle frontal gyrus (MFG) when performing the deception detection tasks. While performing deception detection tasks, infrequent liars showed significantly greater neural activation in the left MFG than the baseline, but frequent liars and innocents did not exhibit this pattern of neural activation in any area of inhibition-related brain regions. The results of individual analysis showed an acceptable accuracy of detecting infrequent liars, but an unacceptable accuracy of detecting frequent liars. These results suggest that using fNIRS monitoring of inhibition-related brain regions is feasible for detecting infrequent liars, for whom deception may be more effortful and therefore more physiologically marked, but not frequent liars.

Motor-commands decoding using peripheral nerve signals: a review

During the last few decades, substantial scientific and technological efforts have been focused on the development of neuroprostheses. The major emphasis has been on techniques for connecting the human nervous system with a robotic prosthesis via natural-feeling interfaces. The peripheral nerves provide access to highly processed and segregated neural command signals from the brain that can in principle be used to determine user intent and control muscles. If these signals could be used, they might allow near-natural and intuitive control of prosthetic limbs with multiple degrees of freedom. This review summarizes the history of neuroprosthetic interfaces and their ability to record from and stimulate peripheral nerves. We also discuss the types of interfaces available and their applications, the kinds of peripheral nerve signals that are used, and the algorithms used to decode them. Finally, we explore the prospects for future development in this area.

Selectivity and Longevity of Peripheral-Nerve and Machine Interfaces: A Review

For those individuals with upper-extremity amputation, a daily normal living activity is no longer possible or it requires additional effort and time. With the aim of restoring their sensory and motor functions, theoretical and technological investigations have been carried out in the field of neuroprosthetic systems. For transmission of sensory feedback, several interfacing modalities including indirect (non-invasive), direct-to-peripheral-nerve (invasive), and cortical stimulation have been applied. Peripheral nerve interfaces demonstrate an edge over the cortical interfaces due to the sensitivity in attaining cortical brain signals. The peripheral nerve interfaces are highly dependent on interface designs and are required to be biocompatible with the nerves to achieve prolonged stability and longevity. Another criterion is the selection of nerves that allows minimal invasiveness and damages as well as high selectivity for a large number of nerve fascicles. In this paper, we review the nerve-machine interface modalities noted above with more focus on peripheral nerve interfaces, which are responsible for provision of sensory feedback. The invasive interfaces for recording and stimulation of electro-neurographic signals include intra-fascicular, regenerative-type interfaces that provide multiple contact channels to a group of axons inside the nerve and the extra-neural-cuff-type interfaces that enable interaction with many axons around the periphery of the nerve. Section Current Prosthetic Technology summarizes the advancements made to date in the field of neuroprosthetics toward the achievement of a bidirectional nerve-machine interface with more focus on sensory feedback. In the Discussion section, the authors propose a hybrid interface technique for achieving better selectivity and long-term stability using the available nerve interfacing techniques.

Functional neural networks of honesty and dishonesty in children: Evidence from graph theory analysis

The present study examined how different brain regions interact with each other during spontaneous honest vs. dishonest communication. More specifically, we took a complex network approach based on the graph-theory to analyze neural response data when children are spontaneously engaged in honest or dishonest acts. Fifty-nine right-handed children between 7 and 12 years of age participated in the study. They lied or told the truth out of their own volition. We found that lying decreased both the global and local efficiencies of children’s functional neural network. This finding, for the first time, suggests that lying disrupts the efficiency of children’s cortical network functioning. Further, it suggests that the graph theory based network analysis is a viable approach to study the neural development of deception.

Evaluation of light detector surface area for functional Near Infrared Spectroscopy

Functional Near Infrared Spectroscopy (fNIRS) is an emerging neuroimaging technique that utilizes near infrared light to detect cortical concentration changes of oxy-hemoglobin and deoxy-hemoglobin non-invasively. Using light sources and detectors over the scalp, multi-wavelength light intensities are recorded as time series and converted to concentration changes of hemoglobin via modified Beer-Lambert law. Here, we describe a potential source for systematic error in the calculation of hemoglobin changes and light intensity measurements. Previous system characterization and analysis studies looked into various fNIRS parameters such as type of light source, number and selection of wavelengths, distance between light source and detector. In this study, we have analyzed the contribution of light detector surface area to the overall outcome. Results from Monte Carlo based digital phantoms indicated that selection of detector area is a critical system parameter in minimizing the error in concentration calculations. The findings here can guide the design of future fNIRS sensors.

Hybrid Brain-Computer Interface Techniques for Improved Classification Accuracy and Increased Number of Commands: A Review

In this article, non-invasive hybrid brain-computer interface (hBCI) technologies for improving classification accuracy and increasing the number of commands are reviewed. Hybridization combining more than two modalities is a new trend in brain imaging and prosthesis control. Electroencephalography (EEG), due to its easy use and fast temporal resolution, is most widely utilized in combination with other brain/non-brain signal acquisition modalities, for instance, functional near infrared spectroscopy (fNIRS), electromyography (EMG), electrooculography (EOG), and eye tracker. Three main purposes of hybridization are to increase the number of control commands, improve classification accuracy and reduce the signal detection time. Currently, such combinations of EEG + fNIRS and EEG + EOG are most commonly employed. Four principal components (i.e., hardware, paradigm, classifiers, and features) relevant to accuracy improvement are discussed. In the case of brain signals, motor imagination/movement tasks are combined with cognitive tasks to increase active brain-computer interface (BCI) accuracy. Active and reactive tasks sometimes are combined: motor imagination with steady-state evoked visual potentials (SSVEP) and motor imagination with P300. In the case of reactive tasks, SSVEP is most widely combined with P300 to increase the number of commands. Passive BCIs, however, are rare. After discussing the hardware and strategies involved in the development of hBCI, the second part examines the approaches used to increase the number of control commands and to enhance classification accuracy. The future prospects and the extension of hBCI in real-time applications for daily life scenarios are provided.

Classification of somatosensory cortex activities using fNIRS

The ability of the somatosensory cortex in differentiating various tactile sensations is very important for a person to perceive the surrounding environment. In this study, we utilize a lab-made multi-channel functional near-infrared spectroscopy (fNIRS) to discriminate the hemodynamic responses (HRs) of four different tactile stimulations (handshake, ball grasp, poking, and cold temperature) applied to the right hand of eight healthy male subjects. The activated brain areas per stimulation are identified with the t-values between the measured data and the desired hemodynamic response function. Linear discriminant analysis is utilized to classify the acquired data into four classes based on three features (mean, peak value, and skewness) of the associated oxy-hemoglobin (HbO) signals. The HRs evoked by the handshake and poking stimulations showed higher peak values in HbO than the ball grasp and cold temperature stimulations. For comparison purposes, additional two-class classifications of poking vs. temperature and handshake vs. ball grasp were performed. The attained classification accuracies were higher than the corresponding chance levels. Our results indicate that fNIRS can be used as an objective measure discriminating different tactile stimulations from the somatosensory cortex of human brain.

Hybrid EEG-fNIRS-Based Eight-Command Decoding for BCI: Application to Quadcopter Control

In this paper, a hybrid electroencephalography–functional near-infrared spectroscopy (EEG–fNIRS) scheme to decode eight active brain commands from the frontal brain region for brain–computer interface is presented. A total of eight commands are decoded by fNIRS, as positioned on the prefrontal cortex, and by EEG, around the frontal, parietal, and visual cortices. Mental arithmetic, mental counting, mental rotation, and word formation tasks are decoded with fNIRS, in which the selected features for classification and command generation are the peak, minimum, and mean ΔHbO values within a 2-s moving window. In the case of EEG, two eyeblinks, three eyeblinks, and eye movement in the up/down and left/right directions are used for four-command generation. The features in this case are the number of peaks and the mean of the EEG signal during 1 s window. We tested the generated commands on a quadcopter in an open space. An average accuracy of 75.6% was achieved with fNIRS for four-command decoding and 86% with EEG for another four-command decoding. The testing results show the possibility of controlling a quadcopter online and in real-time using eight commands from the prefrontal and frontal cortices via the proposed hybrid EEG–fNIRS interface.

Detection and classification of three-class initial dips from prefrontal cortex

In this paper, the use of initial dips using functional near-infrared spectroscopy (fNIRS) for brain-computer interface (BCI) is investigated. Features and window sizes for detecting initial dips are also discussed. Three mental tasks including mental arithmetic, mental counting, and puzzle solving are performed in obtaining fNIRS signals from the prefrontal cortex. Vector-based phase analysis method combined with a threshold circle, as a decision criterion, are used to detect the initial dips. Eight healthy subjects participate in experiment. Linear discriminant analysis is used as a classifier. To classify initial dips, five features (signal mean, peak value, signal slope, skewness, and kurtosis) of oxy-hemoglobin (HbO) and four different window sizes (0~1, 0~1.5, 0~2, and 0~2.5 sec) are examined. It is shown that a combination of signal mean and peak value and a time period of 0~2.5 sec provide the best average classification accuracy of 57.5% for three classes. To further validate the result, three-class classification using the conventional hemodynamic response (HR) is also performed, in which two features (signal mean and signal slope) and 2~7 sec window size have yielded the average classification accuracy of 65.9%. This reveals that fNIRS-based BCI using initial dip detection can reduce the command generation time from 7 sec to 2.5 sec while the classification accuracy is a bit sacrificed from 65.9% to 57.5% for three mental tasks. Further improvement can be made by using deoxy hemoglobin signals in coping with the slow HR problem.

Bundled-Optode Method in Functional Near-Infrared Spectroscopy

In this paper, a theory for detection of the absolute concentrations of oxy-hemoglobin (HbO) and deoxy-hemoglobin (HbR) from hemodynamic responses using a bundled-optode configuration in functional near-infrared spectroscopy (fNIRS) is proposed. The proposed method is then applied to the identification of two fingers (i.e., little and thumb) during their flexion and extension. This experiment involves a continuous-wave-type dual-wavelength (760 and 830 nm) fNIRS and five healthy male subjects. The active brain locations of two finger movements are identified based on the analysis of the t- and p-values of the averaged HbOs, which are quite distinctive. Our experimental results, furthermore, revealed that the hemodynamic responses of two-finger movements are different: The mean, peak, and time-to-peak of little finger movements are higher than those of thumb movements. It is noteworthy that the developed method can be extended to 3-dimensional fNIRS imaging.

Analysis of Different Classification Techniques for Two-Class Functional Near-Infrared Spectroscopy-Based Brain-Computer Interface

We analyse and compare the classification accuracies of six different classifiers for a two-class mental task (mental arithmetic and rest) using functional near-infrared spectroscopy (fNIRS) signals. The signals of the mental arithmetic and rest tasks from the prefrontal cortex region of the brain for seven healthy subjects were acquired using a multichannel continuous-wave imaging system. After removal of the physiological noises, six features were extracted from the oxygenated hemoglobin (HbO) signals. Two- and three-dimensional combinations of those features were used for classification of mental tasks. In the classification, six different modalities, linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), k-nearest neighbour (kNN), the Naïve Bayes approach, support vector machine (SVM), and artificial neural networks (ANN), were utilized. With these classifiers, the average classification accuracies among the seven subjects for the 2- and 3-dimensional combinations of features were 71.6, 90.0, 69.7, 89.8, 89.5, and 91.4% and 79.6, 95.2, 64.5, 94.8, 95.2, and 96.3%, respectively. ANN showed the maximum classification accuracies: 91.4 and 96.3%. In order to validate the results, a statistical significance test was performed, which confirmed that the p values were statistically significant relative to all of the other classifiers (p < 0.005) using HbO signals.

New frontiers in time-domain diffuse optics, a review

The recent developments in time-domain diffuse optics that rely on physical concepts (e.g., time-gating and null distance) and advanced photonic components (e.g., vertical cavity source-emitting laser as light sources, single photon avalanche diode, and silicon photomultipliers as detectors, fast-gating circuits, and time-to-digital converters for acquisition) are focused. This study shows how these tools could lead on one hand to compact and wearable time-domain devices for point-of-care diagnostics down to the consumer level and on the other hand to powerful systems with exceptional depth penetration and sensitivity.

Bundled-optode implementation for 3D imaging in functional near-infrared spectroscopy

The paper presents a functional near-infrared spectroscopy (fNIRS)-based bundled-optode method for detection of the changes of oxy-hemoglobin (HbO) and deoxy-hemoglobin (HbR) concentrations. fNIRS with 32 optodes is utilized to measure five healthy male subjects' brain-hemodynamic responses to arithmetic tasks. Specifically, the coordinates of 256 voxels in the three-dimensional (3D) volume are computed according to the known probe geometry. The mean path length factor in the Beer-Lambert equation is estimated as a function of the emitter-detector distance, which is utilized for computation of the absorption coefficient. The mean values of HbO and HbR obtained from the absorption coefficient are then applied for construction of a 3D fNIRS image. Our results show that the proposed method, as compared with the conventional approach, can detect brain activity with higher spatial resolution. This method can be extended for 3D fNIRS imaging in real-time applications.

Determining Optimal Feature-Combination for LDA Classification of Functional Near-Infrared Spectroscopy Signals in Brain-Computer Interface Application

In this study, we determine the optimal feature-combination for classification of functional near-infrared spectroscopy (fNIRS) signals with the best accuracies for development of a two-class brain-computer interface (BCI). Using a multi-channel continuous-wave imaging system, mental arithmetic signals are acquired from the prefrontal cortex of seven healthy subjects. After removing physiological noises, six oxygenated and deoxygenated hemoglobin (HbO and HbR) features-mean, slope, variance, peak, skewness and kurtosis-are calculated. All possible 2- and 3-feature combinations of the calculated features are then used to classify mental arithmetic vs. rest using linear discriminant analysis (LDA). It is found that the combinations containing mean and peak values yielded significantly higher (p < 0.05) classification accuracies for both HbO and HbR than did all of the other combinations, across all of the subjects. These results demonstrate the feasibility of achieving high classification accuracies using mean and peak values of HbO and HbR as features for classification of mental arithmetic vs. rest for a two-class BCI.

Reduction of Delay in Detecting Initial Dips from Functional Near-Infrared Spectroscopy Signals Using Vector-Based Phase Analysis

In this paper, we present a systematic method to reduce the time lag in detecting initial dips using a vector-based phase diagram and an autoregressive moving average with exogenous signals (ARMAX) model-based [Formula: see text]-step-ahead prediction algorithm. With functional near-infrared spectroscopy (fNIRS), signals related to mental arithmetic and right-hand clenching are acquired from the prefrontal and left primary motor cortices, respectively. The interrelationship between oxygenated hemoglobin, deoxygenated hemoglobin, total hemoglobin and cerebral oxygen exchange are related to initial dips. Specifically, a threshold value from the resting state hemodynamics is incorporated, as a decision criterion, into the vector-based phase diagram to determine the occurrence of initial dips. To further reduce the time lag, a [Formula: see text]-step-ahead prediction method is applied to predict the occurrence of the dips. A combination of the threshold criterion and the prediction method resulted in the delay time of about 0.9[Formula: see text]s. The results demonstrate that rapid detection of initial dip is possible and therefore can be used for real-time brain–computer interfacing.

Decoding four different sound-categories in the auditory cortex using functional near-infrared spectroscopy

The ability of the auditory cortex in the brain to distinguish different sounds is important in daily life. This study investigated whether activations in the auditory cortex caused by different sounds can be distinguished using functional near-infrared spectroscopy (fNIRS). The hemodynamic responses (HRs) in both hemispheres using fNIRS were measured in 18 subjects while exposing them to four sound categories (English-speech, non-English-speech, annoying sounds, and nature sounds). As features for classifying the different signals, the mean, slope, and skewness of the oxy-hemoglobin (HbO) signal were used. With regard to the language-related stimuli, the HRs evoked by understandable speech (English) were observed in a broader brain region than were those evoked by non-English speech. Also, the magnitudes of the HbO signals evoked by English-speech were higher than those of non-English speech. The ratio of the peak values of non-English and English speech was 72.5%. Also, the brain region evoked by annoying sounds was wider than that by nature sounds. However, the signal strength for nature sounds was stronger than that for annoying sounds. Finally, for brain-computer interface (BCI) purposes, the linear discriminant analysis (LDA) and support vector machine (SVM) classifiers were applied to the four sound categories. The overall classification performance for the left hemisphere was higher than that for the right hemisphere. Therefore, for decoding of auditory commands, the left hemisphere is recommended. Also, in two-class classification, the annoying vs. nature sounds comparison provides a higher classification accuracy than the English vs. non-English speech comparison. Finally, LDA performs better than SVM.

Passive BCI based on drowsiness detection: an fNIRS study

We use functional near-infrared spectroscopy (fNIRS) to discriminate the alert and drowsy states for a passive brain-computer interface (BCI). The passive brain signals for the drowsy state are acquired from the prefrontal and dorsolateral prefrontal cortex. The experiment is performed on 13 healthy subjects using a driving simulator, and their brain activity is recorded using a continuous-wave fNIRS system. Linear discriminant analysis (LDA) is employed for training and testing, using the data from the prefrontal, left- and right-dorsolateral prefrontal regions. For classification, eight features are tested: mean oxyhemoglobin, mean deoxyhemoglobin, skewness, kurtosis, signal slope, number of peaks, sum of peaks, and signal peak, in 0~5, 0~10, and 0~15 second time windows, respectively. The results show that the best performance for classification is achieved using mean oxyhemoglobin, the signal peak, and the sum of peaks as features. The average accuracies in the right dorsolateral prefrontal cortex (83.1, 83.4 and 84.9% in the 0~5, 0~10 and 0~15 second time windows, respectively) show that the proposed method has an effective utility for detection of drowsiness for a passive BCI.