Analyzing Brain Functions by Subject Classification of Functional Near-Infrared Spectroscopy Data Using Convolutional Neural Networks Analysis

Convolutional neural networks can detect orthostatic hypotension in Parkinson's disease using resting-state functional near-infrared spectroscopy data

Neurological disorders such as Parkinson's disease (PD) often adversely affect the vascular system, leading to alterations in blood flow patterns. Functional near-infrared spectroscopy (fNIRS) is used to monitor hemodynamic changes via signal measurement. This study investigated the potential of using resting-state fNIRS data through a convolutional neural network (CNN) to evaluate PD with orthostatic hypotension. The CNN demonstrated significant efficacy in analyzing fNIRS data, and it outperformed the other machine learning methods. The results indicate that judicious input data selection can enhance accuracy by over 85%, while including the correlation matrix as an input further improves the accuracy to more than 90%. This study underscores the promising role of CNN-based fNIRS data analysis in the diagnosis and management of the PD. This approach enhances diagnostic accuracy, particularly in resting-state conditions, and can reduce the discomfort and risks associated with current diagnostic methods, such as the head-up tilt test.

Deep neural network to differentiate internet gaming disorder from healthy controls during stop-signal task: a multichannel near-infrared spectroscopy study

Internet Gaming Disorder (IGD), as one of worldwide mental health issues, leads to negative effects on physical and mental health and has attracted public attention. Most studies on IGD are based on screening scales and subjective judgments of doctors, without objective quantitative assessment. However, public understanding of internet gaming disorder lacks objectivity. Therefore, the researches on internet gaming disorder still have many limitations. In this paper, a stop-signal task (SST) was designed to assess inhibitory control in patients with IGD based on prefrontal functional near-infrared spectroscopy (fNIRS). According to the scale, the subjects were divided into health and gaming disorder. A total of 40 subjects (24 internet gaming disorders; 16 healthy controls) signals were used for deep learning-based classification. The seven algorithms used for classification and comparison were deep learning algorithms (DL) and machine learning algorithms (ML), with four and three algorithms in each category, respectively. After applying hold-out method, the performance of the model was verified by accuracy. DL models outperformed traditional ML algorithms. Furthermore, the classification accuracy of the two-dimensional convolution neural network (2D-CNN) was 87.5% among all models. This was the highest accuracy out of all models that were tested. The 2D-CNN was able to outperform the other models due to its ability to learn complex patterns in data. This makes it well-suited for image classification tasks. The findings suggested that a 2D-CNN model is an effective approach for predicting internet gaming disorder. The results show that this is a reliable method with high accuracy to identify patients with IGD and demonstrate that the use of fNIRS to facilitate the development of IGD diagnosis has great potential.

CGAN-rIRN: a data-augmented deep learning approach to accurate classification of mental tasks for a fNIRS-based brain-computer interface

Functional near-infrared spectroscopy (fNIRS) is increasingly used to investigate different mental tasks for brain-computer interface (BCI) control due to its excellent environmental and motion robustness. Feature extraction and classification strategy for fNIRS signal are essential to enhance the classification accuracy of voluntarily controlled BCI systems. The limitation of traditional machine learning classifiers (MLCs) lies in manual feature engineering, which is considered as one of the drawbacks that reduce accuracy. Since the fNIRS signal is a typical multivariate time series with multi-dimensionality and complexity, it makes the deep learning classifier (DLC) ideal for classifying neural activation patterns. However, the inherent bottleneck of DLCs is the requirement of substantial-scale, high-quality labeled training data and expensive computational resources to train deep networks. The existing DLCs for classifying mental tasks do not fully consider the temporal and spatial properties of fNIRS signals. Therefore, a specifically-designed DLC is desired to classify multi-tasks with high accuracy in fNIRS-BCI. To this end, we herein propose a novel data-augmented DLC to accurately classify mental tasks, which employs a convolution-based conditional generative adversarial network (CGAN) for data augmentation and a revised Inception-ResNet (rIRN) based DLC. The CGAN is utilized to generate class-specific synthetic fNIRS signals to augment the training dataset. The network architecture of rIRN is elaborately designed in accordance with the characteristics of the fNIRS signal, with serial multiple spatial and temporal feature extraction modules (FEMs), where each FEM performs deep and multi-scale feature extraction and fusion. The results of the paradigm experiments show that the proposed CGAN-rIRN approach improves the single-trial accuracy for mental arithmetic and mental singing tasks in both the data augmentation and classifier, as compared to the traditional MLCs and the commonly used DLCs. The proposed fully data-driven hybrid deep learning approach paves a promising way to improve the classification performance of volitional control fNIRS-BCI.

Deep learning in fNIRS: a review

Significance: Optical neuroimaging has become a well-established clinical and research tool to monitor cortical activations in the human brain. It is notable that outcomes of functional near-infrared spectroscopy (fNIRS) studies depend heavily on the data processing pipeline and classification model employed. Recently, deep learning (DL) methodologies have demonstrated fast and accurate performances in data processing and classification tasks across many biomedical fields. Aim: We aim to review the emerging DL applications in fNIRS studies. Approach: We first introduce some of the commonly used DL techniques. Then, the review summarizes current DL work in some of the most active areas of this field, including brain-computer interface, neuro-impairment diagnosis, and neuroscience discovery. Results: Of the 63 papers considered in this review, 32 report a comparative study of DL techniques to traditional machine learning techniques where 26 have been shown outperforming the latter in terms of the classification accuracy. In addition, eight studies also utilize DL to reduce the amount of preprocessing typically done with fNIRS data or increase the amount of data via data augmentation. Conclusions: The application of DL techniques to fNIRS studies has shown to mitigate many of the hurdles present in fNIRS studies such as lengthy data preprocessing or small sample sizes while achieving comparable or improved classification accuracy.

Identifying Individuals by fNIRS-Based Brain Functional Network Fingerprints

Individual identification based on brain functional network (BFN) has attracted a lot of research interest in recent years, since it provides a novel biometric for identity authentication, as well as a feasible way of exploring the brain at an individual level. Previous studies have shown that an individual can be identified by its BFN fingerprint estimated from functional magnetic resonance imaging, electroencephalogram, or magnetoencephalography data. Functional near-infrared spectroscopy (fNIRS) is an emerging imaging technique that, by measuring the changes in blood oxygen concentration, can respond to cerebral activities; in this paper, we investigate whether fNIRS-based BFN could be used as a “fingerprint” to identify individuals. In particular, Pearson's correlation is first used to calculate BFN based on the preprocessed fNIRS signals, and then the nearest neighbor scheme is used to match the estimated BFNs between different individuals. Through the experiments on an open-access fNIRS dataset, we have two main findings: (1) under the cases of cross-task (i.e., resting, right-handed, left-handed finger tapping, and foot tapping), the BFN fingerprints generally work well for the individual identification, and, more interestingly, (2) the accuracy under cross-task is well above the accuracy under cross-view (i.e., oxyhemoglobin and de-oxyhemoglobin). These findings indicate that fNIRS-based BFN fingerprint is a potential biometric for identifying individual.

Taking a Deeper Look at the Brain: Predicting Visual Perceptual and Working Memory Load from High-Density fNIRS Data

Predicting workload using physiological sensors has taken on a diffuse set of methods in recent years. However, the majority of these methods train models on small datasets, with small numbers of channel locations on the brain, limiting a models ability to transfer across participants, tasks, or experimental sessions. In this paper, we introduce a new method of modeling a large, cross-participant and cross-session set of high density functional near infrared spectroscopy (fNIRS) data by using an approach grounded in cognitive load theory and employing a Bi-Directional Gated Recurrent Unit (BiGRU) incorporating attention mechanism and self-supervised label augmentation (SLA). We show that our proposed CNN-BiGRU-SLA model can learn and classify different levels of working memory load (WML) and visual processing load (VPL) across participants. Importantly, we leverage a multi-label classification scheme, where our models are trained to predict simultaneously occurring levels of WML and VPL. We evaluate our model using leave-one-participant-out (LOOCV) as well as 10-fold cross validation. Using LOOCV, for binary classification (off/on), we reached an F1-score of 0.9179 for WML and 0.8907 for VPL across 22 participants (each participant did 2 sessions). For multi-level (off, low, high) classification, we reached an F1-score of 0.7972 for WML and 0.7968 for VPL. Using 10-fold cross validation, for multi-level classification, we reached an F1-score of 0.7742 for WML and 0.7741 for VPL.

The Role of Visual Features in Text-Based CAPTCHAs: An fNIRS Study for Usable Security

To mitigate dictionary attacks or similar undesirable automated attacks to information systems, developers mostly prefer using CAPTCHA challenges as Human Interactive Proofs (HIPs) to distinguish between human users and scripts. Appropriate use of CAPTCHA requires a setup that balances between robustness and usability during the design of a challenge. The previous research reveals that most usability studies have used accuracy and response time as measurement criteria for quantitative analysis. The present study aims at applying optical neuroimaging techniques for the analysis of CAPTCHA design. The functional Near-Infrared Spectroscopy technique was used to explore the hemodynamic responses in the prefrontal cortex elicited by CAPTCHA stimulus of varying types. The findings suggest that regions in the left and right dorsolateral and right dorsomedial prefrontal cortex respond to the degrees of line occlusion, rotation, and wave distortions present in a CAPTCHA. The systematic addition of the visual effects introduced nonlinear effects on the behavioral and prefrontal oxygenation measures, indicative of the emergence of Gestalt effects that might have influenced the perception of the overall CAPTCHA figure.

Depression Analysis and Recognition based on Functional Near-infrared Spectroscopy

Depression is the result of a complex interaction of social, psychological and physiological elements. It is now considered to be a major threat to people's physical health, and even as a threat to their lives. Research into the brain disorders of patients suffering from depression can help doctors to understand the pathogenesis of depression and facilitate its diagnosis and treatment. Functional near-infrared spectroscopy (fNIRS) is a non-invasive approach to the detection of brain functions and activities based on changes to the hemoglobin's oxygenation. In this paper, a comprehensive fNIRS-based depression-processing architecture, including the layers of source, feature and model, is first established to guide the deep modeling for fNIRS. In view of the complexity of depression, we propose a methodology in the time and frequency domains for feature extraction and deep neural networks for depression recognition and combining with current research. It is found that compared to non-depressed people, patients with depression have a weaker encephalic area connectivity and lower level of activation in the prefrontal lobe during brain activity. Finally, based on raw data, manual features and channel correlations, to recognize depression, the AlexNet model shows the best performance, especially in terms of the correlation features and presents an accuracy rate of 0.90 and a precision rate of 0.91, which is higher than ResNet18 and machine-learning algorithms on other data. Therefore, the correlation of brain regions can effectively recognize depression (from cases of non-depression), making it significant for the recognition of brain functions in the clinical diagnosis and treatment of depression.

Decoding Three Different Preference Levels of Consumers Using Convolutional Neural Network: A Functional Near-Infrared Spectroscopy Study

This study decodes consumers' preference levels using a convolutional neural network (CNN) in neuromarketing. The classification accuracy in neuromarketing is a critical factor in evaluating the intentions of the consumers. Functional near-infrared spectroscopy (fNIRS) is utilized as a neuroimaging modality to measure the cerebral hemodynamic responses. In this study, a specific decoding structure, called CNN-based fNIRS-data analysis, was designed to achieve a high classification accuracy. Compared to other methods, the automated characteristics, constant training of the dataset, and learning efficiency of the proposed method are the main advantages. The experimental procedure required eight healthy participants (four female and four male) to view commercial advertisement videos of different durations (15, 30, and 60 s). The cerebral hemodynamic responses of the participants were measured. To compare the preference classification performances, CNN was utilized to extract the most common features, including the mean, peak, variance, kurtosis, and skewness. Considering three video durations, the average classification accuracies of 15, 30, and 60 s videos were 84.3, 87.9, and 86.4%, respectively. Among them, the classification accuracy of 87.9% for 30 s videos was the highest. The average classification accuracies of three preferences in females and males were 86.2 and 86.3%, respectively, showing no difference in each group. By comparing the classification performances in three different combinations (like vs. so-so, like vs. dislike, and so-so vs. dislike) between two groups, male participants were observed to have targeted preferences for commercial advertising, and the classification performance 88.4% between "like" vs. "dislike" out of three categories was the highest. Finally, pairwise classification performance are shown as follows: For female, 86.1% (like vs. so-so), 87.4% (like vs. dislike), 85.2% (so-so vs. dislike), and for male 85.7, 88.4, 85.1%, respectively.

Enhanced Accuracy for Multiclass Mental Workload Detection Using Long Short-Term Memory for Brain-Computer Interface

Cognitive workload is one of the widely invoked human factors in the areas of human-machine interaction (HMI) and neuroergonomics. The precise assessment of cognitive and mental workload (MWL) is vital and requires accurate neuroimaging to monitor and evaluate the cognitive states of the brain. In this study, we have decoded four classes of MWL using long short-term memory (LSTM) with 89.31% average accuracy for brain-computer interface (BCI). The brain activity signals are acquired using functional near-infrared spectroscopy (fNIRS) from the prefrontal cortex (PFC) region of the brain. We performed a supervised MWL experimentation with four varying MWL levels on 15 participants (both male and female) and 10 trials of each MWL per participant. Real-time four-level MWL states are assessed using fNIRS system, and initial classification is performed using three strong machine learning (ML) techniques, support vector machine (SVM), k-nearest neighbor (k-NN), and artificial neural network (ANN) with obtained average accuracies of 54.33, 54.31, and 69.36%, respectively. In this study, novel deep learning (DL) frameworks are proposed, which utilizes convolutional neural network (CNN) and LSTM with 87.45 and 89.31% average accuracies, respectively, to solve high-dimensional four-level cognitive states classification problem. Statistical analysis, t-test, and one-way F-test (ANOVA) are also performed on accuracies obtained through ML and DL algorithms. Results show that the proposed DL (LSTM and CNN) algorithms significantly improve classification performance as compared with ML (SVM, ANN, and k-NN) algorithms.

Deep learning a boon for biophotonics?

This review covers original articles using deep learning in the biophotonic field published in the last years. In these years deep learning, which is a subset of machine learning mostly based on artificial neural network geometries, was applied to a number of biophotonic tasks and has achieved state-of-the-art performances. Therefore, deep learning in the biophotonic field is rapidly growing and it will be utilized in the next years to obtain real-time biophotonic decision-making systems and to analyze biophotonic data in general. In this contribution, we discuss the possibilities of deep learning in the biophotonic field including image classification, segmentation, registration, pseudostaining and resolution enhancement. Additionally, we discuss the potential use of deep learning for spectroscopic data including spectral data preprocessing and spectral classification. We conclude this review by addressing the potential applications and challenges of using deep learning for biophotonic data.

Breast Cancer Identification via Thermography Image Segmentation with a Gradient Vector Flow and a Convolutional Neural Network

Breast cancer is the most common cancer among women worldwide with about half a million cases reported each year. Mammary thermography can offer early diagnosis at low cost if adequate thermographic images of the breasts are taken. The identification of breast cancer in an automated way can accelerate many tasks and applications of pathology. This can help complement diagnosis. The aim of this work is to develop a system that automatically captures thermographic images of breast and classifies them as normal and abnormal (without cancer and with cancer). This paper focuses on a segmentation method based on a combination of the curvature function k and the gradient vector flow, and for classification, we proposed a convolutional neural network (CNN) using the segmented breast. The aim of this paper is to compare CNN results with other classification techniques. Thus, every breast is characterized by its shape, colour, and texture, as well as left or right breast. These data were used for training as well as to compare the performance of CNN with three classification techniques: tree random forest (TRF), multilayer perceptron (MLP), and Bayes network (BN). CNN presents better results than TRF, MLP, and BN.

An fNIRS-based investigation of visual merchandising displays for fashion stores

This paper investigates a brain-based approach for visual merchandising display (VMD) in fashion stores. In marketing, VMD has become a research topic of interest. However, VMD research using brain activation information is rare. We examine the hemodynamic responses (HRs) in the prefrontal cortex (PFC) using functional near-infrared spectroscopy (fNIRS) while positive/negative displays of four stores (menswear, womenswear, underwear, and sportswear) are shown to 20 subjects. As features for classifying the HRs, the mean, variance, peak, skewness, kurtosis, t-value, and slope of the signals for a 20-sec time window for the activated channels are analyzed. Linear discriminant analysis is used for classifying two-class (positive and negative displays) and four-class (four fashion stores) models. PFC brain activation maps based on t-values depicting the data from the 16 channels are provided. In the two-class classification, the underwear store had the highest average classification result of 67.04%, whereas the menswear store had the lowest value of 64.15%. Men's classification accuracy for the underwear stores with positive and negative displays was 71.38%, whereas the highest classification accuracy obtained by women for womenswear stores was 73%. The average accuracy over the 20 subjects for positive displays was 50.68%, while that of negative displays was 51.07%. Therefore, these findings suggest that human brain activation is involved in the evaluation of the fashion store displays. It is concluded that fNIRS can be used as a brain-based tool in the evaluation of fashion stores in a daily life environment.