A Unified Analytical Framework With Multiple fNIRS Features for Mental Workload Assessment in the Prefrontal Cortex

Diagnosis of mental disorders using machine learning: Literature review and bibliometric mapping from 2012 to 2023

Background

Mental disorders (MDs) are becoming a leading burden in non-communicable diseases (NCDs). As per the World Health Organization's 2022 assessment report, there was a steep increase of 25 % in MDs during the COVID-19 pandemic. Early diagnosis of MDs can significantly improve treatment outcome and save disability-adjusted life years (DALYs). In recent times, the application of machine learning (ML) and deep learning (DL)) has shown promising results in the diagnosis of MDs, and the field has witnessed a huge research output in the form of research publications. Therefore, a bibliometric mapping along with a review of recent advancements is required.

Methods

This study presents a bibliometric analysis and review of the research, published over the last 10 years. Literature searches were conducted in the Scopus database for the period from January 1, 2012, to June 9, 2023. The data was filtered and screened to include only relevant and reliable publications. A total of 2811 journal articles were found. The data was exported to a comma-separated value (CSV) format for further analysis. Furthermore, a review of 40 selected studies was performed.

Results

The popularity of ML techniques in diagnosing MDs has been growing, with an annual research growth rate of 17.05 %. The Journal of Affective Disorders published the most documents (n = 97), while Wang Y. (n = 64) has published the most articles. Lotka's law is observed, with a minority of authors contributing the majority of publications. The top affiliating institutes are the West China Hospital of Sichuan University followed by the University of California, with China and the US dominating the top 10 institutes. While China has more publications, papers affiliated with the US receive more citations. Depression and schizophrenia are the primary focuses of ML and deep learning (DL) in mental disease detection. Co-occurrence network analysis reveals that ML is associated with depression, schizophrenia, autism, anxiety, ADHD, obsessive-compulsive disorder, and PTSD. Popular algorithms include support vector machine (SVM) classifier, decision tree classifier, and random forest classifier. Furthermore, DL is linked to neuroimaging techniques such as MRI, fMRI, and EEG, as well as bipolar disorder. Current research trends encompass DL, LSTM, generalized anxiety disorder, feature fusion, and convolutional neural networks.

Numerical Solution of Inverse Problem in Functional Near Infrared Spectroscopy using L1-Norm Method

It has been more than three decades since researchers began investigating functional near-infrared spectroscopy (fNIRs) and its applications with near-infrared light for use in both clinical and pre-clinical settings. In order to increase the accuracy of fNIRs of complex tissue structures, it is necessary to create more advanced image reconstruction methods. Real fNIRs data have been used to develop an implementation of the L1-Norm approach for tackling the inverse problem in this work. The Monte Carlo (MC) simulation is used to construct the sensitivity matrix for this research. Finally, a numerical algorithm for the L1-Norm approach of image reconstruction is developed and implemented in MATLAB to aid in the process. The results showed good agreement with the actual fNIRs data.

Impacts of complex electromagnetic radiation and low-frequency noise exposure conditions on the cognitive function of operators

Background

Both electromagnetic radiation (EMR) and low-frequency noise (LFN) are widespread and influential environmental factors, and operators are inevitably exposed to both EMR and LFN within a complex exposure environment. The potential adverse effects of such exposure on human health must be considered seriously. This study aimed to investigate the effects of EMR and LFN on cognitive function as well as their interaction effect, which remain unclear.

Methods

Sixty young male college students were randomly grouped and experiments were conducted with a 2 × 2 factorial design in a shielded chamber. Mental workload (MWL) levels of the study subjects were measured and assessed using the NASA-task load index (TLX) subjective scale, an n-back task paradigm, and the functional near-infrared spectroscopy (fNIRS) imaging technique.

Results

For the 3-back task, the NASA-TLX subjective scale revealed a statistically significant main effect of LFN intensity, which enhanced the subjects' MWL level (F = 8.716, p < 0.01). Behavioral performance revealed that EMR intensity (430.1357 MHz, 10.75 W/m²) and LFN intensity (0-200 Hz, 72.9 dB) had a synergistic interaction effect, and the correct response time was statistically significantly prolonged by the combined exposure (F = 4.343, p < 0.05). The fNIRS imaging technique revealed a synergistic interaction effect between operational EMR intensity and operational LFN intensity, with statistically significant effects on the activation levels in the left and right dorsolateral prefrontal cortex (DLPFC). The mean β values of DLPFC were significantly increased (L-DLPFC F = 5.391, p < 0.05, R-DLPFC F = 4.222, p < 0.05), and the relative concentrations of oxyhemoglobin in the DLPFC were also significantly increased (L-DLPFC F = 4.925, p < 0.05, R-DLPFC F = 9.715, p < 0.01).

Conclusion

We found a statistically significant interaction effect between EMR (430.1357 MHz, 10.75 W/m²) and LFN (0-200 Hz, 72.9 dB) when simultaneously exposing subjects to both for 30 min. We conclude that exposure to this complex environment can cause a statistically significant increase in the MWL level of operators, and even alterations in their cognitive function.

Combining robust level extraction and unsupervised adaptive classification for high-accuracy fNIRS-BCI: An evidence on single-trial differentiation between mentally arithmetic- and singing-tasks

Functional near-infrared spectroscopy (fNIRS) is a safe and non-invasive optical imaging technique that is being increasingly used in brain-computer interfaces (BCIs) to recognize mental tasks. Unlike electroencephalography (EEG) which directly measures neural activation, fNIRS signals reflect neurovascular-coupling inducing hemodynamic response that can be slow in time and varying in the pattern. The established classifiers extend the EEG-ones by mostly employing the feature based supervised models such as the support vector machine (SVM) and linear discriminant analysis (LDA), and fail to timely characterize the level-sensitive hemodynamic pattern. A dedicated classifier is desired for intentional activity recognition of fNIRS-BCI, including the adaptive acquisition of response relevant features and accurate discrimination of implied ideas. To this end, we herein propose a specifically-designed joint adaptive classification method that combines a Kalman filtering (KF) for robust level extraction and an adaptive Gaussian mixture model (a-GMM) for enhanced pattern recognition. The simulative investigations and paradigm experiments have shown that the proposed KF/a-GMM classification method can effectively track the random variations of task-evoked brain activation patterns, and improve the accuracy of single-trial classification task of mental arithmetic vs. mental singing, as compared to the conventional methods, e.g., those that employ combinations of the band-pass filtering (BPF) based feature extractors (mean, slope, and variance, etc.) and the classical recognizers (GMM, SVM, and LDA). The proposed approach paves a promising way for developing the real-time fNIRS-BCI technique.

EEG/fNIRS Based Workload Classification Using Functional Brain Connectivity and Machine Learning

There is high demand for techniques to estimate human mental workload during some activities for productivity enhancement or accident prevention. Most studies focus on a single physiological sensing modality and use univariate methods to analyse multi-channel electroencephalography (EEG) data. This paper proposes a new framework that relies on the features of hybrid EEG-functional near-infrared spectroscopy (EEG-fNIRS), supported by machine-learning features to deal with multi-level mental workload classification. Furthermore, instead of the well-used univariate power spectral density (PSD) for EEG recording, we propose using bivariate functional brain connectivity (FBC) features in the time and frequency domains of three bands: delta (0.5-4 Hz), theta (4-7 Hz) and alpha (8-15 Hz). With the assistance of the fNIRS oxyhemoglobin and deoxyhemoglobin (HbO and HbR) indicators, the FBC technique significantly improved classification performance at a 77% accuracy for 0-back vs. 2-back and 83% for 0-back vs. 3-back using a public dataset. Moreover, topographic and heat-map visualisation indicated that the distinguishing regions for EEG and fNIRS showed a difference among the 0-back, 2-back and 3-back test results. It was determined that the best region to assist the discrimination of the mental workload for EEG and fNIRS is different. Specifically, the posterior area performed the best for the posterior midline occipital (POz) EEG in the alpha band and fNIRS had superiority in the right frontal region (AF8).

Decoding declarative memory process for predicting memory retrieval based on source localization

Many studies have focused on understanding memory processes due to their importance in daily life. Differences in timing and power spectra of brain signals during encoding task have been linked to later remembered items and were recently used to predict memory retrieval performance. However, accuracies remain low when using non-invasive methods for acquiring brain signals, mainly due to the low spatial resolution. This study investigates the prediction of successful retrieval using estimated source activity corresponding either to cortical or subcortical structures through source localization. Electroencephalogram (EEG) signals were recorded while participants performed a declarative memory task. Frequency-time analysis was performed using signals from encoding and retrieval tasks to confirm the importance of neural oscillations and their relationship with later remembered and forgotten items. Significant differences in the power spectra between later remembered and forgotten items were found before and during the presentation of the stimulus in the encoding task. Source activity estimation revealed differences in the beta band power over the medial parietal and medial prefrontal areas prior to the presentation of the stimulus, and over the cuneus and lingual areas during the presentation of the stimulus. Additionally, there were significant differences during the stimuli presentation during the retrieval task. Prediction of later remembered items was performed using surface potentials and estimated source activity. The results showed that source localization increases classification performance compared to the one using surface potentials. These findings support the importance of incorporating spatial features of neural activity to improve the prediction of memory retrieval.

Prefrontal Activation During Effortful Processing Differentiates Memory Abilities in Adults with Memory Complaints

Background: Identifying individuals at increased risks for developing Alzheimer’s disease (AD) is crucial for early intervention. Memory complaints are associated with brain abnormalities characteristic of AD in cognitively normal older people. However, the utility of memory complaints for predicting mild cognitive impairment (MCI) or AD onset remains controversial, likely due to the heterogeneous nature of this construct. Objective: We investigated whether prefrontal oxygenation changes measured by functional near-infrared spectroscopy (fNIRS) during an arduous cognitive task, previously shown to be associated with the AD syndrome, could differentiate memory abilities among individuals with memory complaints. Episodic memory performance was adopted as a proxy for MCI/AD risks since it has been shown to predict AD progression across stages. Methods: Thirty-six adults self-reporting memory complaints in the absence of memory impairment completed a verbal list learning test and underwent a digit n-back paradigm with an easy (0-back) and a difficult (2-back) condition. K-means clustering was applied to empirically derive memory complaint subgroups based on fNIRS-based prefrontal oxygenation changes during the effortful 2-back task. Results: Cluster analysis revealed two subgroups characterized by high (n = 12) and low (n = 24) bilateral prefrontal activation during the 2-back but not a 0-back task. The low activation group was significantly less accurate across the n-back task and recalled significantly fewer words on the verbal memory test compared to the high activation group. Conclusion: fNIRS may have the potential to differentiate verbal memory abilities in individuals with self-reported memory complaints.

FGANet: fNIRS-guided Attention Network for Hybrid EEG-fNIRS Brain-Computer Interfaces

Non-invasive brain-computer interfaces (BCIs) have been widely used for neural decoding, linking neural signals to control devices. Hybrid BCI systems using electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) have received significant attention for overcoming the limitations of EEG- and fNIRS-standalone BCI systems. However, most hybrid EEG-fNIRS BCI studies have focused on late fusion because of discrepancies in their temporal resolutions and recording locations. Despite the enhanced performance of hybrid BCIs, late fusion methods have difficulty in extracting correlated features in both EEG and fNIRS signals. Therefore, in this study, we proposed a deep learning-based early fusion structure, which combines two signals before the fully-connected layer, called the fNIRS-guided attention network (FGANet). First, 1D EEG and fNIRS signals were converted into 3D EEG and fNIRS tensors to spatially align EEG and fNIRS signals at the same time point. The proposed fNIRS-guided attention layer extracted a joint representation of EEG and fNIRS tensors based on neurovascular coupling, in which the spatially important regions were identified from fNIRS signals, and detailed neural patterns were extracted from EEG signals. Finally, the final prediction was obtained by weighting the sum of the prediction scores of the EEG and fNIRS-guided attention features to alleviate performance degradation owing to delayed fNIRS response. In the experimental results, the FGANet significantly outperformed the EEG-standalone network. Furthermore, the FGANet has 4.0% and 2.7% higher accuracy than the state-of-the-art algorithms in mental arithmetic and motor imagery tasks, respectively.

Inter-Brain Synchrony Levels According to Task Execution Modes and Difficulty Levels: an fNIRS/GSR Study

Hyperscanning is a brain imaging technique that measures brain synchrony caused by social interactions. Recent research on hyperscanning has revealed substantial inter-brain synchrony (IBS), but little is known about the link between IBS and mental workload. To study this link, we conducted an experiment consisting of button-pressing tasks of three different difficulty levels for the cooperation and competition modes with 56 participants aged 23.7±3.8 years (mean±standard deviation). We attempted to observe IBS using functional near-infrared spectroscopy (fNIRS) and galvanic skin response (GSR) to assess the activities of the human autonomic nervous system. We found that the IBS levels increased in a frequency band of 0.075-0.15 Hz, which was unrelated to the task repetition frequency in the cooperation mode according to the task difficulty level. Significant relative inter-brain synchrony (RIBS) increases were observed in three and 10 channels out of 15 for the hard tasks compared to the normal and easy tasks, respectively. We observed that the average GSR values increased with increasing task difficulty levels for the competition mode only. Thus, our results suggest that the IBS revealed by fNIRS and GSR is not related to the hemodynamic changes induced by mental workload, simple behavioral synchrony such as button-pressing timing, or autonomic nervous system activity. IBS is thus explicitly caused by social interactions such as cooperation.

Subdividing Stress Groups into Eustress and Distress Groups Using Laterality Index Calculated from Brain Hemodynamic Response

A stress group should be subdivided into eustress (low-stress) and distress (high-stress) groups to better evaluate personal cognitive abilities and mental/physical health. However, it is challenging because of the inconsistent pattern in brain activation. We aimed to ascertain the necessity of subdividing the stress groups. The stress group was screened by salivary alpha-amylase (sAA) and then, the brain’s hemodynamic reactions were measured by functional near-infrared spectroscopy (fNIRS) based on the near-infrared biosensor. We compared the two stress subgroups categorized by sAA using a newly designed emotional stimulus-response paradigm with an international affective picture system (IAPS) to enhance hemodynamic signals induced by the target effect. We calculated the laterality index for stress (LIS) from the measured signals to identify the dominantly activated cortex in both the subgroups. Both the stress groups exhibited brain activity in the right frontal cortex. Specifically, the eustress group exhibited the largest brain activity, whereas the distress group exhibited recessive brain activity, regardless of positive or negative stimuli. LIS values were larger in the order of the eustress, control, and distress groups; this indicates that the stress group can be divided into eustress and distress groups. We built a foundation for subdividing stress groups into eustress and distress groups using fNIRS.

Anger Experience and Anger Expression Through Drawing in Schizophrenia: An fNIRS Study

Differences in emotion experience and emotion expression between patients with schizophrenia and the healthy population have long been the focus of research and clinical attention. However, few empirical studies have addressed this topic using art-making as a tool of emotion expression. This study explores the differences in brain mechanism during the process of expressing anger between patients with schizophrenia and healthy participants using pictographic psychological techniques. We used functional near-infrared spectroscopy to fully detect changes in frontal cortex activity among participants in two groups-schizophrenia and healthy-during the process of experiencing and expressing anger. The results showed that there were no differences in the experience of anger between the two groups. In the process of anger expression, the dorsolateral prefrontal cortex, frontal pole, and other regions showed significant negative activation among patients with schizophrenia, which was significantly different from that of the healthy group. There were significant differences between patients with schizophrenia and the healthy group in the drawing features, drawing contents, and the ability to describe the contents of their drawings. Moreover, the effect size of the latter was greater than those of the former two. In terms of emotion expression, the drawing data and brain activation data were significantly correlated in each group; however, the correlation patterns differed between groups.

Optimizing Mental Workload Estimation by Detecting Baseline State Using Vector Phase Analysis Approach

Improper baseline return from the previous task-evoked hemodynamic response (HR) can contribute to a large variation in the subsequent HR, affecting the estimation of mental workload in brain-computer interface systems. In this study, we proposed a method using vector phase analysis to detect the baseline state as being optimal or suboptimal. We hypothesize that selecting neuronal-related HR as observed in the optimal-baseline blocks can lead to an improvement in estimating mental workload. Oxygenated and deoxygenated hemoglobin concentration changes were integrated as parts of the vector phase. The proposed method was applied to a block-design functional near-infrared spectroscopy dataset (total blocks = 1384), measured on 24 subjects performing multiple difficulty levels of mental arithmetic task. Significant differences in hemodynamic signal change were observed between the optimal- and suboptimal-baseline blocks detected using the proposed method. This supports the effectiveness of the proposed method in detecting baseline state for better estimation of mental workload. The results further highlight the need of customized recovery duration. In short, the proposed method offers a practical approach to detect task-evoked signals, without the need of extra probes.