An Augmented-Reality fNIRS-Based Brain-Computer Interface: A Proof-of-Concept Study

Functional Near-Infrared Spectrometry as a Useful Diagnostic Tool for Understanding the Visual System: A Review

This comprehensive review explores the role of Functional Near-Infrared Spectroscopy (fNIRS) in advancing our understanding of the visual system. Beginning with an introduction to fNIRS, we delve into its historical development, highlighting how this technology has evolved over time. The core of the review critically examines the advantages and disadvantages of fNIRS, offering a balanced view of its capabilities and limitations in research and clinical settings. We extend our discussion to the diverse applications of fNIRS beyond its traditional use, emphasizing its versatility across various fields. In the context of the visual system, this review provides an in-depth analysis of how fNIRS contributes to our understanding of eye function, including eye diseases. We discuss the intricacies of the visual cortex, how it responds to visual stimuli and the implications of these findings in both health and disease. A unique aspect of this review is the exploration of the intersection between fNIRS, virtual reality (VR), augmented reality (AR) and artificial intelligence (AI). We discuss how these cutting-edge technologies are synergizing with fNIRS to open new frontiers in visual system research. The review concludes with a forward-looking perspective, envisioning the future of fNIRS in a rapidly evolving technological landscape and its potential to revolutionize our approach to studying and understanding the visual system.

Metaverse Wearables for Immersive Digital Healthcare: A Review

The recent exponential growth of metaverse technology has been instrumental in reshaping a myriad of sectors, not least digital healthcare. This comprehensive review critically examines the landscape and future applications of metaverse wearables toward immersive digital healthcare. The key technologies and advancements that have spearheaded the metamorphosis of metaverse wearables are categorized, encapsulating all-encompassed extended reality, such as virtual reality, augmented reality, mixed reality, and other haptic feedback systems. Moreover, the fundamentals of their deployment in assistive healthcare (especially for rehabilitation), medical and nursing education, and remote patient management and treatment are investigated. The potential benefits of integrating metaverse wearables into healthcare paradigms are multifold, encompassing improved patient prognosis, enhanced accessibility to high-quality care, and high standards of practitioner instruction. Nevertheless, these technologies are not without their inherent challenges and untapped opportunities, which span privacy protection, data safeguarding, and innovation in artificial intelligence. In summary, future research trajectories and potential advancements to circumvent these hurdles are also discussed, further augmenting the incorporation of metaverse wearables within healthcare infrastructures in the post-pandemic era.

It takes two (seconds): decreasing encoding time for two-choice functional near-infrared spectroscopy brain-computer interface communication

Significance

Brain-computer interfaces (BCIs) can provide severely motor-impaired patients with a motor-independent communication channel. Functional near-infrared spectroscopy (fNIRS) constitutes a promising BCI-input modality given its high mobility, safety, user comfort, cost-efficiency, and relatively low motion sensitivity.

Aim

The present study aimed at developing an efficient and convenient two-choice fNIRS communication BCI by implementing a relatively short encoding time (2 s), considerably increasing communication speed, and decreasing the cognitive load of BCI users.

Approach

To encode binary answers to 10 biographical questions, 10 healthy adults repeatedly performed a combined motor-speech imagery task within 2 different time windows guided by auditory instructions. Each answer-encoding run consisted of 10 trials. Answers were decoded during the ongoing experiment from the time course of the individually identified most-informative fNIRS channel-by-chromophore combination.

Results

The answers of participants were decoded online with an accuracy of 85.8% (run-based group mean). Post-hoc analysis yielded an average single-trial accuracy of 68.1%. Analysis of the effect of number of trial repetitions showed that the best information-transfer rate could be obtained by combining four encoding trials.

Conclusions

The study demonstrates that an encoding time as short as 2 s can enable immediate, efficient, and convenient fNIRS-BCI communication.

Explainable artificial intelligence model to predict brain states from fNIRS signals

Objective: Most Deep Learning (DL) methods for the classification of functional Near-Infrared Spectroscopy (fNIRS) signals do so without explaining which features contribute to the classification of a task or imagery. An explainable artificial intelligence (xAI) system that can decompose the Deep Learning mode's output onto the input variables for fNIRS signals is described here. Approach: We propose an xAI-fNIRS system that consists of a classification module and an explanation module. The classification module consists of two separately trained sliding window-based classifiers, namely, (i) 1-D Convolutional Neural Network (CNN); and (ii) Long Short-Term Memory (LSTM). The explanation module uses SHAP (SHapley Additive exPlanations) to explain the CNN model's output in terms of the model's input. Main results: We observed that the classification module was able to classify two types of datasets: (a) Motor task (MT), acquired from three subjects; and (b) Motor imagery (MI), acquired from 29 subjects, with an accuracy of over 96% for both CNN and LSTM models. The explanation module was able to identify the channels contributing the most to the classification of MI or MT and therefore identify the channel locations and whether they correspond to oxy- or deoxy-hemoglobin levels in those locations. Significance: The xAI-fNIRS system can distinguish between the brain states related to overt and covert motor imagery from fNIRS signals with high classification accuracy and is able to explain the signal features that discriminate between the brain states of interest.

Performance comparison of systemic activity correction in functional near-infrared spectroscopy for methods with and without short distance channels

Significance: Functional near-infrared spectroscopy (fNIRS) is a promising tool for neurofeedback (NFB) or brain-computer interfaces (BCIs). However, fNIRS signals are typically highly contaminated by systemic activity (SA) artifacts, and, if not properly corrected, NFB or BCIs run the risk of being based on noise instead of brain activity. This risk can likely be reduced by correcting for SA, in particular when short-distance channels (SDCs) are available. Literature comparing correction methods with and without SDCs is still sparse, specifically comparisons considering single trials are lacking. Aim: This study aimed at comparing the performance of SA correction methods with and without SDCs. Approach: Semisimulated and real motor task data of healthy older adults were used. Correction methods without SDCs included a simple and a more advanced spatial filter. Correction methods with SDCs included a regression approach considering only the closest SDC and two GLM-based methods, one including all eight SDCs and one using only two a priori selected SDCs as regressors. All methods were compared with data uncorrected for SA and correction performance was assessed with quality measures quantifying signal improvement and spatial specificity at single trial level. Results: All correction methods were found to improve signal quality and enhance spatial specificity as compared with the uncorrected data. Methods with SDCs usually outperformed methods without SDCs. Correction methods without SDCs tended to overcorrect the data. However, the exact pattern of results and the degree of differences observable between correction methods varied between semisimulated and real data, and also between quality measures. Conclusions: Overall, results confirmed that both Δ [ HbO ] and Δ [ HbR ] are affected by SA and that correction methods with SDCs outperform methods without SDCs. Nonetheless, improvements in signal quality can also be achieved without SDCs and should therefore be given priority over not correcting for SA.

Research on Rehabilitation Training Strategies Using Multimodal Virtual Scene Stimulation

It is difficult for stroke patients with flaccid paralysis to receive passive rehabilitation training. Therefore, virtual rehabilitation technology that integrates the motor imagery brain-computer interface and virtual reality technology has been applied to the field of stroke rehabilitation and has evolved into a physical rehabilitation training method. This virtual rehabilitation technology can enhance the initiative and adaptability of patient rehabilitation. To maximize the deep activation of the subjects motor nerves and accelerate the remodeling mechanism of motor nerve function, this study designed a brain-computer interface rehabilitation training strategy using different virtual scenes, including static scenes, dynamic scenes, and VR scenes. Including static scenes, dynamic scenes, and VR scenes. We compared and analyzed the degree of neural activation and the recognition rate of motor imagery in stroke patients after motor imagery training using stimulation of different virtual scenes, The results show that under the three scenarios, The order of degree of neural activation and the recognition rate of motor imagery from high to low is: VR scenes, dynamic scenes, static scenes. This paper provided the research basis for a virtual rehabilitation strategy that could integrate the motor imagery brain-computer interface and virtual reality technology.

See, Hear, or Feel - to Speak: A Versatile Multiple-Choice Functional Near-Infrared Spectroscopy-Brain-Computer Interface Feasible With Visual, Auditory, or Tactile Instructions

Severely motor-disabled patients, such as those suffering from the so-called "locked-in" syndrome, cannot communicate naturally. They may benefit from brain-computer interfaces (BCIs) exploiting brain signals for communication and therewith circumventing the muscular system. One BCI technique that has gained attention recently is functional near-infrared spectroscopy (fNIRS). Typically, fNIRS-based BCIs allow for brain-based communication via voluntarily modulation of brain activity through mental task performance guided by visual or auditory instructions. While the development of fNIRS-BCIs has made great progress, the reliability of fNIRS-BCIs across time and environments has rarely been assessed. In the present fNIRS-BCI study, we tested six healthy participants across three consecutive days using a straightforward four-choice fNIRS-BCI communication paradigm that allows answer encoding based on instructions using various sensory modalities. To encode an answer, participants performed a motor imagery task (mental drawing) in one out of four time periods. Answer encoding was guided by either the visual, auditory, or tactile sensory modality. Two participants were tested outside the laboratory in a cafeteria. Answers were decoded from the time course of the most-informative fNIRS channel-by-chromophore combination. Across the three testing days, we obtained mean single- and multi-trial (joint analysis of four consecutive trials) accuracies of 62.5 and 85.19%, respectively. Obtained multi-trial accuracies were 86.11% for visual, 80.56% for auditory, and 88.89% for tactile sensory encoding. The two participants that used the fNIRS-BCI in a cafeteria obtained the best single- (72.22 and 77.78%) and multi-trial accuracies (100 and 94.44%). Communication was reliable over the three recording sessions with multi-trial accuracies of 86.11% on day 1, 86.11% on day 2, and 83.33% on day 3. To gauge the trade-off between number of optodes and decoding accuracy, averaging across two and three promising fNIRS channels was compared to the one-channel approach. Multi-trial accuracy increased from 85.19% (one-channel approach) to 91.67% (two-/three-channel approach). In sum, the presented fNIRS-BCI yielded robust decoding results using three alternative sensory encoding modalities. Further, fNIRS-BCI communication was stable over the course of three consecutive days, even in a natural (social) environment. Therewith, the developed fNIRS-BCI demonstrated high flexibility, reliability and robustness, crucial requirements for future clinical applicability.

Guiding functional near-infrared spectroscopy optode-layout design using individual (f)MRI data: effects on signal strength

Significance: Designing optode layouts is an essential step for functional near-infrared spectroscopy (fNIRS) experiments as the quality of the measured signal and the sensitivity to cortical regions-of-interest depend on how optodes are arranged on the scalp. This becomes particularly relevant for fNIRS-based brain-computer interfaces (BCIs), where developing robust systems with few optodes is crucial for clinical applications. Aim: Available resources often dictate the approach researchers use for optode-layout design. We investigated whether guiding optode layout design using different amounts of subject-specific magnetic resonance imaging (MRI) data affects the fNIRS signal quality and sensitivity to brain activation when healthy participants perform mental-imagery tasks typically used in fNIRS-BCI experiments. Approach: We compared four approaches that incrementally incorporated subject-specific MRI information while participants performed mental-calculation, mental-rotation, and inner-speech tasks. The literature-based approach (LIT) used a literature review to guide the optode layout design. The probabilistic approach (PROB) employed individual anatomical data and probabilistic maps of functional MRI (fMRI)-activation from an independent dataset. The individual fMRI (iFMRI) approach used individual anatomical and fMRI data, and the fourth approach used individual anatomical, functional, and vascular information of the same subject (fVASC). Results: The four approaches resulted in different optode layouts and the more informed approaches outperformed the minimally informed approach (LIT) in terms of signal quality and sensitivity. Further, PROB, iFMRI, and fVASC approaches resulted in a similar outcome. Conclusions: We conclude that additional individual MRI data lead to a better outcome, but that not all the modalities tested here are required to achieve a robust setup. Finally, we give preliminary advice to efficiently using resources for developing robust optode layouts for BCI and neurofeedback applications.