Estimation of general linear model coefficients for real-time application

Real-time and Recursive Estimators for Functional MRI Quality Assessment

Real-time quality assessment (rtQA) of functional magnetic resonance imaging (fMRI) based on blood oxygen level-dependent (BOLD) signal changes is critical for neuroimaging research and clinical applications. The losses of BOLD sensitivity because of different types of technical and physiological noise remain major sources of fMRI artifacts. Due to difficulty of subjective visual perception of image distortions during data acquisitions, a comprehensive automatic rtQA is needed. To facilitate rapid rtQA of fMRI data, we applied real-time and recursive quality assessment methods to whole-brain fMRI volumes, as well as time-series of target brain areas and resting-state networks. We estimated recursive temporal signal-to-noise ratio (rtSNR) and contrast-to-noise ratio (rtCNR), and real-time head motion parameters by a framewise rigid-body transformation (translations and rotations) using the conventional current to template volume registration. In addition, we derived real-time framewise (FD) and micro (MD) displacements based on head motion parameters and evaluated the temporal derivative of root mean squared variance over voxels (DVARS). For monitoring time-series of target regions and networks, we estimated the number of spikes and amount of filtered noise by means of a modified Kalman filter. Finally, we applied the incremental general linear modeling (GLM) to evaluate real-time contributions of nuisance regressors (linear trend and head motion). Proposed rtQA was demonstrated in real-time fMRI neurofeedback runs without and with excessive head motion and real-time simulations of neurofeedback and resting-state fMRI data. The rtQA was implemented as an extension of the open-source OpenNFT software written in Python, MATLAB and C++ for neurofeedback, task-based, and resting-state paradigms. We also developed a general Python library to unify real-time fMRI data processing and neurofeedback applications. Flexible estimation and visualization of rtQA facilitates efficient rtQA of fMRI data and helps the robustness of fMRI acquisitions by means of substantiating decisions about the necessity of the interruption and re-start of the experiment and increasing the confidence in neural estimates.

Improving Attention through Individualized fNIRS Neurofeedback Training: A Pilot Study

Attention is a particularly important indicator in life, as inattention can lead to many negative consequences. As a non-invasive intervention, real-time neurofeedback training can effectively enhance individuals’ attention adjustment abilities. However, previous studies have neglected to consider differences among individuals. In this study, an individualized neurofeedback training (INT) method based on functional near-infrared spectroscopy (fNIRS) was proposed for attention improvement and compared with non-individualized neurofeedback training (NINT). The neurofeedback channels and thresholds were determined individually for each subject. Then, participants conducted four runs of neurofeedback training. Two attention tests (i.e., AX version of continuous performance task (AX-CPT) and attention network test (ANT)) were used to assess the performance of the neurofeedback training. The length of time that the two groups of participants continuously kept their oxygenated hemoglobin concentration above a threshold showed an increasing trend, and the improvement rate of the INT group was higher than that of the NINT group. The reaction times for both groups showed a downward trend, but the INT group declined more significantly. In the fNIRS data, it was observed that the activation degree of the INT group in the middle and dorsolateral prefrontal areas was higher than that of the NINT group. It is preliminarily proved that the proposed INT method can effectively improve the attention level, and its overall performance is better than that of the NINT method.

A Library for fMRI Real-Time Processing Systems in Python (RTPSpy) With Comprehensive Online Noise Reduction, Fast and Accurate Anatomical Image Processing, and Online Processing Simulation

Real-time fMRI (rtfMRI) has enormous potential for both mechanistic brain imaging studies or treatment-oriented neuromodulation. However, the adaption of rtfMRI has been limited due to technical difficulties in implementing an efficient computational framework. Here, we introduce a python library for real-time fMRI (rtfMRI) data processing systems, Real-Time Processing System in python (RTPSpy), to provide building blocks for a custom rtfMRI application with extensive and advanced functionalities. RTPSpy is a library package including (1) a fast, comprehensive, and flexible online fMRI image processing modules comparable to offline denoising, (2) utilities for fast and accurate anatomical image processing to define an anatomical target region, (3) a simulation system of online fMRI processing to optimize a pipeline and target signal calculation, (4) simple interface to an external application for feedback presentation, and (5) a boilerplate graphical user interface (GUI) integrating operations with RTPSpy library. The fast and accurate anatomical image processing utility wraps external tools, including FastSurfer, ANTs, and AFNI, to make tissue segmentation and region of interest masks. We confirmed that the quality of the output masks was comparable with FreeSurfer, and the anatomical image processing could complete in a few minutes. The modular nature of RTPSpy provides the ability to use it for a simulation analysis to optimize a processing pipeline and target signal calculation. We present a sample script for building a real-time processing pipeline and running a simulation using RTPSpy. The library also offers a simple signal exchange mechanism with an external application using a TCP/IP socket. While the main components of the RTPSpy are the library modules, we also provide a GUI class for easy access to the RTPSpy functions. The boilerplate GUI application provided with the package allows users to develop a customized rtfMRI application with minimum scripting labor. The limitations of the package as it relates to environment-specific implementations are discussed. These library components can be customized and can be used in parts. Taken together, RTPSpy is an efficient and adaptable option for developing rtfMRI applications. Code available at: https://github.com/mamisaki/RTPSpy.

Real-Time Functional MRI in the Treatment of Mental Health Disorders

Multiple mental disorders have been associated with dysregulation of precise brain processes. However, few therapeutic approaches can correct such specific patterns of brain activity. Since the late 1960s and early 1970s, many researchers have hoped that this feat could be achieved by closed-loop brain imaging approaches, such as neurofeedback, that aim to modulate brain activity directly. However, neurofeedback never gained mainstream acceptance in mental health, in part due to methodological considerations. In this review, we argue that, when contemporary methodological guidelines are followed, neurofeedback is one of the few intervention methods in psychology that can be assessed in double-blind placebo-controlled trials. Furthermore, using new advances in machine learning and statistics, it is now possible to target very precise patterns of brain activity for therapeutic purposes. We review the recent literature in functional magnetic resonance imaging neurofeedback and discuss current and future applications to mental health.

Expected final online publication date for the Annual Review of Clinical Psychology, Volume 18 is May 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

The impact of real-time fMRI denoising on online evaluation of brain activity and functional connectivity

Objective. Comprehensive denoising is imperative in functional magnetic resonance imaging (fMRI) analysis to reliably evaluate neural activity from the blood oxygenation level dependent signal. In real-time fMRI, however, only a minimal denoising process has been applied and the impact of insufficient denoising on online brain activity estimation has not been assessed comprehensively. This study evaluated the noise reduction performance of online fMRI processes in a real-time estimation of regional brain activity and functional connectivity.Approach.We performed a series of real-time processing simulations of online fMRI processing, including slice-timing correction, motion correction, spatial smoothing, signal scaling, and noise regression with high-pass filtering, motion parameters, motion derivatives, global signal, white matter/ventricle average signals, and physiological noise models with image-based retrospective correction of physiological motion effects (RETROICOR) and respiration volume per time (RVT).Main results.All the processing was completed in less than 400 ms for whole-brain voxels. Most processing had a benefit for noise reduction except for RVT that did not work due to the limitation of the online peak detection. The global signal regression, white matter/ventricle signal regression, and RETROICOR had a distinctive noise reduction effect, depending on the target signal, and could not substitute for each other. Global signal regression could eliminate the noise-associated bias in the mean dynamic functional connectivity across time.Significance.The results indicate that extensive real-time denoising is possible and highly recommended for real-time fMRI applications.

Effect of Acupuncture Stimulation of Hegu (LI4) and Taichong (LR3) on the Resting-State Networks in Alzheimer's Disease: Beyond the Default Mode Network

It was reported that acupuncture could treat Alzheimer's disease (AD) with the potential mechanisms remaining unclear. The aim of the study is to explore the effect of the combination stimulus of Hegu (LI4) and Taichong (LR3) on the resting-state brain networks in AD, beyond the default network (DMN). Twenty-eight subjects including 14 AD patients and 14 healthy controls (HCs) matched by age, gender, and educational level were recruited in this study. After the baseline resting-state MRI scans, the manual acupuncture stimulation was performed for 3 minutes, and then, another 10 minutes of resting-state fMRI scans was acquired. In addition to the DMN, five other resting-state networks were identified by independent component analysis (ICA), including left frontal parietal network (lFPN), right frontal parietal network (rFPN), visual network (VN), sensorimotor network (SMN), and auditory network (AN). And the impaired connectivity in the lFPN, rFPN, SMN, and VN was found in AD patients compared with those in HCs. After acupuncture, significantly decreased connectivity in the right middle frontal gyrus (MFG) of rFPN (P = 0.007) was identified in AD patients. However, reduced connectivity in the right inferior frontal gyrus (IFG) (P = 0.047) and left superior frontal gyrus (SFG) (P = 0.041) of lFPN and some regions of the SMN (the left inferior parietal lobula (P = 0.004), left postcentral gyrus (PoCG) (P = 0.001), right PoCG (P = 0.032), and right MFG (P = 0.010)) and the right MOG of VN (P = 0.003) was indicated in HCs. In addition, after controlling for the effect of acupuncture on HCs, the functional connectivity of the right cerebellum crus I, left IFG, and left angular gyrus (AG) of lFPN showed to be decreased, while the left MFG of IFPN and the right lingual gyrus of VN increased in AD patients. These findings might have some reference values for the interpretation of the combination stimulus of Hegu (LI4) and Taichong (LR3) in AD patients, which could deepen our understanding of the potential mechanisms of acupuncture on AD.

Quality and denoising in real-time functional magnetic resonance imaging neurofeedback: A methods review

Neurofeedback training using real-time functional magnetic resonance imaging (rtfMRI-NF) allows subjects voluntary control of localised and distributed brain activity. It has sparked increased interest as a promising non-invasive treatment option in neuropsychiatric and neurocognitive disorders, although its efficacy and clinical significance are yet to be determined. In this work, we present the first extensive review of acquisition, processing and quality control methods available to improve the quality of the neurofeedback signal. Furthermore, we investigate the state of denoising and quality control practices in 128 recently published rtfMRI-NF studies. We found: (a) that less than a third of the studies reported implementing standard real-time fMRI denoising steps, (b) significant room for improvement with regards to methods reporting and (c) the need for methodological studies quantifying and comparing the contribution of denoising steps to the neurofeedback signal quality. Advances in rtfMRI-NF research depend on reproducibility of methods and results. Notably, a systematic effort is needed to build up evidence that disentangles the various mechanisms influencing neurofeedback effects. To this end, we recommend that future rtfMRI-NF studies: (a) report implementation of a set of standard real-time fMRI denoising steps according to a proposed COBIDAS-style checklist (https://osf.io/kjwhf/), (b) ensure the quality of the neurofeedback signal by calculating and reporting community-informed quality metrics and applying offline control checks and (c) strive to adopt transparent principles in the form of methods and data sharing and support of open-source rtfMRI-NF software. Code and data for reproducibility, as well as an interactive environment to explore the study data, can be accessed at https://github.com/jsheunis/quality-and-denoising-in-rtfmri-nf.

Real-time fMRI for brain-computer interfacing

Brain-computer interfaces (BCIs) based on functional magnetic resonance imaging (fMRI) provide an important complement to other noninvasive BCIs. While fMRI has several disadvantages (being nonportable, methodologically challenging, costly, and noisy), it is the only method providing high spatial resolution whole-brain coverage of brain activation. These properties allow relating mental activities to specific brain regions and networks providing a transparent scheme for BCI users to encode information and for real-time fMRI BCI systems to decode the intents of the user. Various mental activities have been used successfully in fMRI BCIs so far that can be classified into the four categories: (a) higher-order cognitive tasks (e.g., mental calculation), (b) covert language-related tasks (e.g., mental speech and mental singing), (c) imagery tasks (motor, visual, auditory, tactile, and emotion imagery), and (d) selective attention tasks (visual, auditory, and tactile attention). While the ultimate spatial and temporal resolution of fMRI BCIs is limited by the physiologic properties of the hemodynamic response, technical and analytical advances will likely lead to substantially improved fMRI BCIs in the future using, for example, decoding of imagined letter shapes at 7T as the basis for more "natural" communication BCIs.

No time for drifting: Comparing performance and applicability of signal detrending algorithms for real-time fMRI

As a consequence of recent technological advances in the field of functional magnetic resonance imaging (fMRI), results can now be made available in real-time. This allows for novel applications such as online quality assurance of the acquisition, intra-operative fMRI, brain-computer-interfaces, and neurofeedback. To that aim, signal processing algorithms for real-time fMRI must reliably correct signal contaminations due to physiological noise, head motion, and scanner drift. The aim of this study was to compare performance of the commonly used online detrending algorithms exponential moving average (EMA), incremental general linear model (iGLM) and sliding window iGLM (iGLMwindow). For comparison, we also included offline detrending algorithms (i.e., MATLAB's and SPM8's native detrending functions). Additionally, we optimized the EMA control parameter, by assessing the algorithm's performance on a simulated data set with an exhaustive set of realistic experimental design parameters. First, we optimized the free parameters of the online and offline detrending algorithms. Next, using simulated data, we systematically compared the performance of the algorithms with respect to varying levels of Gaussian and colored noise, linear and non-linear drifts, spikes, and step function artifacts. Additionally, using in vivo data from an actual rt-fMRI experiment, we validated our results in a post hoc offline comparison of the different detrending algorithms. Quantitative measures show that all algorithms perform well, even though they are differently affected by the different artifact types. The iGLM approach outperforms the other online algorithms and achieves online detrending performance that is as good as that of offline procedures. These results may guide developers and users of real-time fMRI analyses tools to best account for the problem of signal drifts in real-time fMRI.

Neu3CA-RT: A framework for real-time fMRI analysis

Real-time functional magnetic resonance imaging (rtfMRI) allows visualisation of ongoing brain activity of the subject in the scanner. Denoising algorithms aim to rid acquired data of confounding effects, enhancing the blood oxygenation level-dependent (BOLD) signal. Further image processing and analysis methods, like general linear models (GLM) or multivariate analysis, then present application-specific information to the researcher. These processes are typically applied to regions of interest but, increasingly, rtfMRI techniques extract and classify whole brain functional networks and dynamics as correlates for brain states or behaviour, particularly in neuropsychiatric and neurocognitive disorders. We present Neu³CA-RT: a Matlab-based rtfMRI analysis framework aiming to advance scientific knowledge on real-time cognitive brain activity and to promote its translation into clinical practice. Design considerations are listed based on reviewing existing rtfMRI approaches. The toolbox integrates established SPM preprocessing routines, real-time GLM mapping of fMRI data to a basis set of spatial brain networks, correlation of activity with 50 behavioural profiles from the BrainMap database, and an intuitive user interface. The toolbox is demonstrated in a task-based experiment where a subject executes visual, auditory and motor tasks inside a scanner. In three out of four experiments, resulting behavioural profiles agreed with the expected brain state.

Existence of Initial Dip for BCI: An Illusion or Reality

A tight coupling between the neuronal activity and the cerebral blood flow (CBF) is the motivation of many hemodynamic response (HR)-based neuroimaging modalities. The increase in neuronal activity causes the increase in CBF that is indirectly measured by HR modalities. Upon functional stimulation, the HR is mainly categorized in three durations: (i) initial dip, (ii) conventional HR (i.e., positive increase in HR caused by an increase in the CBF), and (iii) undershoot. The initial dip is a change in oxygenation prior to any subsequent increase in CBF and spatially more specific to the site of neuronal activity. Despite additional evidence from various HR modalities on the presence of initial dip in human and animal species (i.e., cat, rat, and monkey); the existence/occurrence of an initial dip in HR is still under debate. This article reviews the existence and elusive nature of the initial dip duration of HR in intrinsic signal optical imaging (ISOI), functional magnetic resonance imaging (fMRI), and functional near-infrared spectroscopy (fNIRS). The advent of initial dip and its elusiveness factors in ISOI and fMRI studies are briefly discussed. Furthermore, the detection of initial dip and its role in brain-computer interface using fNIRS is examined in detail. The best possible application for the initial dip utilization and its future implications using fNIRS are provided.

2-CHANNEL CONVOLUTIONAL 3D DEEP NEURAL NETWORK (2CC3D) FOR FMRI ANALYSIS: ASD CLASSIFICATION AND FEATURE LEARNING

In this paper, we propose a new whole brain fMRI-analysis scheme to identify autism spectrum disorder (ASD) and explore biological markers in ASD classification. To utilize both spatial and temporal information in fMRI, our method investigates the potential benefits of using a sliding window over time to measure temporal statistics (mean and standard deviation) and using 3D convolutional neural networks (CNNs) to capture spatial features. The sliding window created 2-channel images, which were used as inputs to the 3D CNN. From the outputs of the 3D CNN convolutional layers, ASD related fMRI spatial features were directly deciphered. Input formats and sliding window parameters were investigated in our study. The power of aligning 2-channel images was shown in our proposed method. Compared with traditional machine learning classification models, our proposed 2CC3D method increased mean F-scores over 8.5%.

A Novel Feature-Map Based ICA Model for Identifying the Individual, Intra/Inter-Group Brain Networks across Multiple fMRI Datasets

Independent component analysis (ICA) has been widely used in functional magnetic resonance imaging (fMRI) data analysis to evaluate functional connectivity of the brain; however, there are still some limitations on ICA simultaneously handling neuroimaging datasets with diverse acquisition parameters, e.g., different repetition time, different scanner, etc. Therefore, it is difficult for the traditional ICA framework to effectively handle ever-increasingly big neuroimaging datasets. In this research, a novel feature-map based ICA framework (FMICA) was proposed to address the aforementioned deficiencies, which aimed at exploring brain functional networks (BFNs) at different scales, e.g., the first level (individual subject level), second level (intragroup level of subjects within a certain dataset) and third level (intergroup level of subjects across different datasets), based only on the feature maps extracted from the fMRI datasets. The FMICA was presented as a hierarchical framework, which effectively made ICA and constrained ICA as a whole to identify the BFNs from the feature maps. The simulated and real experimental results demonstrated that FMICA had the excellent ability to identify the intergroup BFNs and to characterize subject-specific and group-specific difference of BFNs from the independent component feature maps, which sharply reduced the size of fMRI datasets. Compared with traditional ICAs, FMICA as a more generalized framework could efficiently and simultaneously identify the variant BFNs at the subject-specific, intragroup, intragroup-specific and intergroup levels, implying that FMICA was able to handle big neuroimaging datasets in neuroscience research.

OpenNFT: An open-source Python/Matlab framework for real-time fMRI neurofeedback training based on activity, connectivity and multivariate pattern analysis

Neurofeedback based on real-time functional magnetic resonance imaging (rt-fMRI) is a novel and rapidly developing research field. It allows for training of voluntary control over localized brain activity and connectivity and has demonstrated promising clinical applications. Because of the rapid technical developments of MRI techniques and the availability of high-performance computing, new methodological advances in rt-fMRI neurofeedback become possible. Here we outline the core components of a novel open-source neurofeedback framework, termed Open NeuroFeedback Training (OpenNFT), which efficiently integrates these new developments. This framework is implemented using Python and Matlab source code to allow for diverse functionality, high modularity, and rapid extendibility of the software depending on the user's needs. In addition, it provides an easy interface to the functionality of Statistical Parametric Mapping (SPM) that is also open-source and one of the most widely used fMRI data analysis software. We demonstrate the functionality of our new framework by describing case studies that include neurofeedback protocols based on brain activity levels, effective connectivity models, and pattern classification approaches. This open-source initiative provides a suitable framework to actively engage in the development of novel neurofeedback approaches, so that local methodological developments can be easily made accessible to a wider range of users.

How to Build a Hybrid Neurofeedback Platform Combining EEG and fMRI

Multimodal neurofeedback estimates brain activity using information acquired with more than one neurosignal measurement technology. In this paper we describe how to set up and use a hybrid platform based on simultaneous electroencephalography (EEG) and functional magnetic resonance imaging (fMRI), then we illustrate how to use it for conducting bimodal neurofeedback experiments. The paper is intended for those willing to build a multimodal neurofeedback system, to guide them through the different steps of the design, setup, and experimental applications, and help them choose a suitable hardware and software configuration. Furthermore, it reports practical information from bimodal neurofeedback experiments conducted in our lab. The platform presented here has a modular parallel processing architecture that promotes real-time signal processing performance and simple future addition and/or replacement of processing modules. Various unimodal and bimodal neurofeedback experiments conducted in our lab showed high performance and accuracy. Currently, the platform is able to provide neurofeedback based on electroencephalography and functional magnetic resonance imaging, but the architecture and the working principles described here are valid for any other combination of two or more real-time brain activity measurement technologies.

WASICA: An effective wavelet-shrinkage based ICA model for brain fMRI data analysis

BACKGROUND

Researches declared that the super-Gaussian property contributed to the success of some spatial independent component analysis (ICA) algorithms in brain fMRI source separation (e.g., Infomax and FastICA), which implied that sparse approximation transforming the sources (super-Gaussian or Gaussian-like) with stronger super-Gaussian feature would possibly improve the separation performance of these algorithms.

NEW METHOD

This paper presented a novel wavelet-shrinkage based ICA (WASICA) model, an extension of our previous SACICA, for single-subject analysis. In contrast, two main aspects had been effectively enhanced: (1) sparse approximation coefficients set formation, made up of two sub-procedures: the wavelet-shrinkage of wavelet packet (WP) tree nodes, and the automatic nodes selection and integration based on the relative WP energy; (2) ICA-based decomposition and reconstruction, composed of temporal dynamics extraction using ICA, WP reconstruction based on the sparse approximation coefficients set and least-square-based functional networks reconstruction.

RESULTS

The wavelet-shrinkage and the automatic nodes selection and integration simultaneously transformed both the mixtures and underlying sources into effective sparse approximation coefficients sets, exhibiting stronger super-Gaussian distribution; WP projected-back approximation with nuisance-exclusion contributed to networks reconstruction.

COMPARISON WITH EXISTING METHODS

Simulation 1 revealed WASICA successfully recovered super-Gaussian and some Gaussian-like sources. Simulation 2 and hybrid data experiments showed that WASICA with good temporal performance had stronger source recovery ability and signal detection sensitivity spatially than FastICA, Infomax and SACICA did; the higher intra-consistency in task-related experiments denoted WASICA occupied stronger spatial robustness against smooth kernels.

CONCLUSIONS

WASICA was a promising brain signal separation model with charming spatial-temporal performance.

Quantitative ultrasound facilitates the exploration of morphological association of the long head biceps tendon with supraspinatus tendon full thickness tear

Backgrounds

Pathology of the long head biceps tendon (LHBT) is associated with rotator cuff tears but whether the LHBT texture changes following supraspinatus tendon full thickness tear (SSFT) can be detected at the extra-articular segment remains unknown. This cross-sectional study aimed to explore the morphological differences of the LHBT in shoulders with and without deficient rotator cuffs by using quantitative ultrasound.

Materials and Methods

We selected 145 cases with SSFT and 145 age-and- gender-matched controls. The width, thickness, flattening ratio, cross-sectional area, and echogenicity ratio of the LHBT were measured and a general linear model was used to clarify the relationship between rotator cuff pathology and LHBT morphology. The receiver operating characteristic curves of each parameter were constructed for SSFT discrimination and the maximal Youden indexes were used to define the best cut-off points.

Results

We found increased thickness and cross-sectional area but decreased flattening ratio in shoulders with SSFT, and no between-group differences in the width and echogenicity ratio. The LHBT appearance was modified by biceps peritendinous effusion and medial subluxation, but not by the size of SSFT. The flattening ratio was the best discriminator for SSFT with an area under curve of 0.81 (95% confidence interval, 0.76–0.86). The cut-off values to differentiate between the non-tear and tear groups were 2.00 mm of the thickness, 1.73 of the flattening ratio and 10.53 mm² of the cross-sectional area.

Conclusion

Quantitative ultrasound facilitated the detection of the LHBT morphological changes following SSFT and demonstrated its potential utility in discriminating rotator cuff deficiency.

Temporally and spatially constrained ICA of fMRI data analysis

Constrained independent component analysis (CICA) is capable of eliminating the order ambiguity that is found in the standard ICA and extracting the desired independent components by incorporating prior information into the ICA contrast function. However, the current CICA method produces constraints that are based on only one type of prior information (temporal/spatial), which may increase the dependency of CICA on the accuracy of the prior information. To improve the robustness of CICA and to reduce the impact of the accuracy of prior information on CICA, we proposed a temporally and spatially constrained ICA (TSCICA) method that incorporated two types of prior information, both temporal and spatial, as constraints in the ICA. The proposed approach was tested using simulated fMRI data and was applied to a real fMRI experiment using 13 subjects who performed a movement task. Additionally, the performance of TSCICA was compared with the ICA method, the temporally CICA (TCICA) method and the spatially CICA (SCICA) method. The results from the simulation and from the real fMRI data demonstrated that TSCICA outperformed TCICA, SCICA and ICA in terms of robustness to noise. Moreover, the TSCICA method displayed better robustness to prior temporal/spatial information than the TCICA/SCICA method.

Incremental activation detection for real-time fMRI series using robust Kalman filter

Real-time functional magnetic resonance imaging (rt-fMRI) is a technique that enables us to observe human brain activations in real time. However, some unexpected noises that emerged in fMRI data collecting, such as acute swallowing, head moving and human manipulations, will cause much confusion and unrobustness for the activation analysis. In this paper, a new activation detection method for rt-fMRI data is proposed based on robust Kalman filter. The idea is to add a variation to the extended kalman filter to handle the additional sparse measurement noise and a sparse noise term to the measurement update step. Hence, the robust Kalman filter is designed to improve the robustness for the outliers and can be computed separately for each voxel. The algorithm can compute activation maps on each scan within a repetition time, which meets the requirement for real-time analysis. Experimental results show that this new algorithm can bring out high performance in robustness and in real-time activation detection.

Online decoding of object-based attention using real-time fMRI

Visual attention is used to selectively filter relevant information depending on current task demands and goals. Visual attention is called object-based attention when it is directed to coherent forms or objects in the visual field. This study used real-time functional magnetic resonance imaging for moment-to-moment decoding of attention to spatially overlapped objects belonging to two different object categories. First, a whole-brain classifier was trained on pictures of faces and places. Subjects then saw transparently overlapped pictures of a face and a place, and attended to only one of them while ignoring the other. The category of the attended object, face or place, was decoded on a scan-by-scan basis using the previously trained decoder. The decoder performed at 77.6% accuracy indicating that despite competing bottom-up sensory input, object-based visual attention biased neural patterns towards that of the attended object. Furthermore, a comparison between different classification approaches indicated that the representation of faces and places is distributed rather than focal. This implies that real-time decoding of object-based attention requires a multivariate decoding approach that can detect these distributed patterns of cortical activity.

High-throughput optogenetic functional magnetic resonance imaging with parallel computations

Optogenetic functional magnetic resonance imaging (of MRI) technology enables cell-type-specific, temporally precise neuronal control and the accurate, in vivo readout of the resulting activity across the entire brain. With the ability to precisely control excitation and inhibition parameters and accurately record the resulting activity, there is an increased need for a high-throughput method to bring the of MRI studies to their full potential. In this paper, an advanced system facilitating real-time fMRI with interactive control and analysis in a fraction of the MRI acquisition repetition time (TR) is proposed. With high-processing speed, sufficient time will be available for the integration of future developments that further enhance of MRI data or streamline the study. We designed and implemented a highly optimised, massively parallel system using graphics processing units (GPUs), which achieves the reconstruction, motion correction, and analysis of 3D volume data in approximately 12.80 ms. As a result, with a 750 ms TR and 4 interleaf fMRI acquisition, we can now conduct sliding window reconstruction, motion correction, analysis and display in approximately 1.7% of the TR. Therefore, a significant amount of time can now be allocated to integrating advanced but computationally intensive methods that improve image quality and enhance the analysis results within a TR. Utilising the proposed high-throughput imaging platform with sliding window reconstruction, we were also able to observe the much-debated initial dips in our of MRI data. Combined with methods to further improve SNR, the proposed system will enable efficient real-time, interactive, high-throughput of MRI studies.

Navigation of a telepresence robot via covert visuospatial attention and real-time fMRI

Brain-computer interfaces (BCIs) allow people with severe neurological impairment and without ability to control their muscles to regain some control over their environment. The BCI user performs a mental task to regulate brain activity, which is measured and translated into commands controlling some external device. We here show that healthy participants are capable of navigating a robot by covertly shifting their visuospatial attention. Covert Visuospatial Attention (COVISA) constitutes a very intuitive brain function for spatial navigation and does not depend on presented stimuli or on eye movements. Our robot is equipped with motors and a camera that sends visual feedback to the user who can navigate it from a remote location. We used an ultrahigh field MRI scanner (7 Tesla) to obtain fMRI signals that were decoded in real time using a support vector machine. Four healthy subjects with virtually no training succeeded in navigating the robot to at least three of four target locations. Our results thus show that with COVISA BCI, realtime robot navigation can be achieved. Since the magnitude of the fMRI signal has been shown to correlate well with the magnitude of spectral power changes in the gamma frequency band in signals measured by intracranial electrodes, the COVISA concept may in future translate to intracranial application in severely paralyzed people.

Real-time fMRI applied to pain management

Current views recognize the brain as playing a pivotal role in the arising and maintenance of pain experience. Real-time fMRI (rtfMRI) feedback is a potential tool for pain modulation that directly targets the brain with the goal of restoring regulatory function. Though still relatively new, rtfMRI is a rapidly developing technology that has evolved in the last 15 years from simple proof of concept experiments to demonstrations of learned control of single and multiple brain areas. Numerous studies indicate rtfMRI feedback assisted control over specific brain areas may have applications including mood regulation, language processing, neurorehabilitation in stroke, enhancement of perception and learning, and pain management. We discuss in detail earlier work from our lab in which rtfMRI feedback was used to train both healthy controls and chronic pain patients to modulate anterior cingulate cortex (ACC) activation for the purposes of altering pain experience. Both groups improved in their ability to control ACC activation and modulate their pain with rtfMRI feedback training. Furthermore, the degree to which participants were able to modulate their pain correlated with the degree of control over ACC activation. We additionally review current advances in rtfMRI feedback, such as real-time pattern classification, that bring the technology closer to more comprehensive control over neural function. Finally, remaining methodological questions concerning the further development of rtfMRI feedback and its implications for the future of pain research are also discussed.

Neurofeedback using real-time near-infrared spectroscopy enhances motor imagery related cortical activation

Accumulating evidence indicates that motor imagery and motor execution share common neural networks. Accordingly, mental practices in the form of motor imagery have been implemented in rehabilitation regimes of stroke patients with favorable results. Because direct monitoring of motor imagery is difficult, feedback of cortical activities related to motor imagery (neurofeedback) could help to enhance efficacy of mental practice with motor imagery. To determine the feasibility and efficacy of a real-time neurofeedback system mediated by near-infrared spectroscopy (NIRS), two separate experiments were performed. Experiment 1 was used in five subjects to evaluate whether real-time cortical oxygenated hemoglobin signal feedback during a motor execution task correlated with reference hemoglobin signals computed off-line. Results demonstrated that the NIRS-mediated neurofeedback system reliably detected oxygenated hemoglobin signal changes in real-time. In Experiment 2, 21 subjects performed motor imagery of finger movements with feedback from relevant cortical signals and irrelevant sham signals. Real neurofeedback induced significantly greater activation of the contralateral premotor cortex and greater self-assessment scores for kinesthetic motor imagery compared with sham feedback. These findings suggested the feasibility and potential effectiveness of a NIRS-mediated real-time neurofeedback system on performance of kinesthetic motor imagery. However, these results warrant further clinical trials to determine whether this system could enhance the effects of mental practice in stroke patients.

Enhancement of temporal resolution and BOLD sensitivity in real-time fMRI using multi-slab echo-volumar imaging

In this study, a new approach to high-speed fMRI using multi-slab echo-volumar imaging (EVI) is developed that minimizes geometrical image distortion and spatial blurring, and enables nonaliased sampling of physiological signal fluctuation to increase BOLD sensitivity compared to conventional echo-planar imaging (EPI). Real-time fMRI using whole brain 4-slab EVI with 286 ms temporal resolution (4mm isotropic voxel size) and partial brain 2-slab EVI with 136 ms temporal resolution (4×4×6 mm(3) voxel size) was performed on a clinical 3 Tesla MRI scanner equipped with 12-channel head coil. Four-slab EVI of visual and motor tasks significantly increased mean (visual: 96%, motor: 66%) and maximum t-score (visual: 263%, motor: 124%) and mean (visual: 59%, motor: 131%) and maximum (visual: 29%, motor: 67%) BOLD signal amplitude compared with EPI. Time domain moving average filtering (2s width) to suppress physiological noise from cardiac and respiratory fluctuations further improved mean (visual: 196%, motor: 140%) and maximum (visual: 384%, motor: 200%) t-scores and increased extents of activation (visual: 73%, motor: 70%) compared to EPI. Similar sensitivity enhancement, which is attributed to high sampling rate at only moderately reduced temporal signal-to-noise ratio (mean: -52%) and longer sampling of the BOLD effect in the echo-time domain compared to EPI, was measured in auditory cortex. Two-slab EVI further improved temporal resolution for measuring task-related activation and enabled mapping of five major resting state networks (RSNs) in individual subjects in 5 min scans. The bilateral sensorimotor, the default mode and the occipital RSNs were detectable in time frames as short as 75 s. In conclusion, the high sampling rate of real-time multi-slab EVI significantly improves sensitivity for studying the temporal dynamics of hemodynamic responses and for characterizing functional networks at high field strength in short measurement times.

Real-time decoding of brain responses to visuospatial attention using 7T fMRI

Brain-Computer interface technologies mean to create new communication channels between our mind and our environment, independent of the motor system, by detecting and classifying self regulation of local brain activity. BCIs can provide patients with severe paralysis a means to communicate and to live more independent lives. There has been a growing interest in using invasive recordings for BCI to improve the signal quality. This also potentially gives access to new control strategies previously inaccessible by non-invasive methods. However, before surgery, the best implantation site needs to be determined. The blood-oxygen-level dependent signal changes measured with fMRI have been shown to agree well spatially with those found with invasive electrodes, and are the best option for pre-surgical localization. We show, using real-time fMRI at 7T, that eye movement-independent visuospatial attention can be used as a reliable control strategy for BCIs. At this field strength even subtle signal changes can be detected in single trials thanks to the high contrast-to-noise ratio. A group of healthy subjects were instructed to move their attention between three (two peripheral and one central) spatial target regions while keeping their gaze fixated at the center. The activated regions were first located and thereafter the subjects were given real-time feedback based on the activity in these regions. All subjects managed to regulate local brain areas without training, which suggests that visuospatial attention is a promising new target for intracranial BCI. ECoG data recorded from one epilepsy patient showed that local changes in gamma-power can be used to separate the three classes.

Real-time fMRI: a tool for local brain regulation

Real-time fMRI permits simultaneous measurement and observation of brain activity during an ongoing task. One of the most challenging applications of real-time fMRI in neuroscientific and clinical research is the possibility of acquiring volitional control of localized brain activity using real-time fMRI-based neurofeedback protocols. Real-time fMRI allows the experimenter to noninvasively manipulate brain activity as an independent variable to observe the effects on behavior. Real-time fMRI neurofeedback studies demonstrated that learned control of the local brain activity leads to specific changes in behavior. Here, the authors describe the implementation and application of real-time fMRI with particular emphasis on the self-regulation of local brain activity and the investigation of brain-function relationships. Real-time fMRI represents a promising new approach to cognitive neuroscience that could complement traditional neuroimaging techniques by providing more causal insights into the functional role of circumscribed brain regions in behavior.

Real-time functional MRI using pseudo-continuous arterial spin labeling

The first implementation of real-time acquisition and analysis of arterial spin labeling-based functional magnetic resonance imaging time series is presented in this article. The implementation uses a pseudo-continuous labeling scheme followed by a spiral k-space acquisition trajectory. Real-time reconstruction of the images, preprocessing, and regression analysis of the functional magnetic resonance imaging data were implemented on a laptop computer interfaced with the MRI scanner. The method allows the user to track the current raw data, subtraction images, and the cumulative t-statistic map overlaid on a cumulative subtraction image. The user is also able to track the time course of individual time courses and interactively selects a region of interest as a nuisance covariate. The pulse sequence allows the user to adjust acquisition and labeling parameters while observing their effect on the image within two successive pulse repetition times. This method is demonstrated by two functional imaging experiments: a simultaneous finger-tapping and visual stimulation paradigm, and a bimanual finger-tapping task.

Kalman estimator- and general linear model-based on-line brain activation mapping by near-infrared spectroscopy

BACKGROUND

Near-infrared spectroscopy (NIRS) is a non-invasive neuroimaging technique that recently has been developed to measure the changes of cerebral blood oxygenation associated with brain activities. To date, for functional brain mapping applications, there is no standard on-line method for analysing NIRS data.

METHODS

In this paper, a novel on-line NIRS data analysis framework taking advantages of both the general linear model (GLM) and the Kalman estimator is devised. The Kalman estimator is used to update the GLM coefficients recursively, and one critical coefficient regarding brain activities is then passed to a t-statistical test. The t-statistical test result is used to update a topographic brain activation map. Meanwhile, a set of high-pass filters is plugged into the GLM to prevent very low-frequency noises, and an autoregressive (AR) model is used to prevent the temporal correlation caused by physiological noises in NIRS time series. A set of data recorded in finger tapping experiments is studied using the proposed framework.

RESULTS

The obtained results suggest that the method can effectively track the task related brain activation areas, and prevent the noise distortion in the estimation while the experiment is running. Thereby, the potential of the proposed method for real-time NIRS-based brain imaging was demonstrated.

CONCLUSIONS

This paper presents a novel on-line approach for analysing NIRS data for functional brain mapping applications. This approach demonstrates the potential of a real-time-updating topographic brain activation map.

Computing moment-to-moment BOLD activation for real-time neurofeedback

Estimating moment-to-moment changes in blood oxygenation level dependent (BOLD) activation levels from functional magnetic resonance imaging (fMRI) data has applications for learned regulation of regional activation, brain state monitoring, and brain-machine interfaces. In each of these contexts, accurate estimation of the BOLD signal in as little time as possible is desired. This is a challenging problem due to the low signal-to-noise ratio of fMRI data. Previous methods for real-time fMRI analysis have either sacrificed the ability to compute moment-to-moment activation changes by averaging several acquisitions into a single activation estimate or have sacrificed accuracy by failing to account for prominent sources of noise in the fMRI signal. Here we present a new method for computing the amount of activation present in a single fMRI acquisition that separates moment-to-moment changes in the fMRI signal intensity attributable to neural sources from those due to noise, resulting in a feedback signal more reflective of neural activation. This method computes an incremental general linear model fit to the fMRI time series, which is used to calculate the expected signal intensity at each new acquisition. The difference between the measured intensity and the expected intensity is scaled by the variance of the estimator in order to transform this residual difference into a statistic. Both synthetic and real data were used to validate this method and compare it to the only other published real-time fMRI method.

fMRI data visualization with BrainBlend and Blender

The visualization and exploration of neuroimaging data is important for the analysis of anatomical and functional magnetic resonance (MR) images and thresholded statistical parametric maps. While two-dimensional orthogonal views of neuroimaging data are used to display statistical analyses, real three-dimensional (3d) depictions are helpful for showing the spatial distribution of a functional network, as well as its temporal evolution. However, viewers that are freely available on the internet offer only limited rendering capabilities and depictions of temporal changes of the blood oxygen level-dependent (BOLD) response. In this article, we present BrainBlend, a toolbox for the software package Statistical Parametric Mapping (SPM), that generates voxeldata files to be used with the open-source 3d-software "Blender". Our interface between SPM and Blender permits the use of any Analyze- and Nifti-file for the creation of images and animations of transparent volumetric objects. Different kinds of anatomical, functional and statistical data can be rendered as volumetric objects in order to convey an immediate understanding of the three-dimensional shape. Representations of functional networks can be animated using a time course extracted from the general linear model or the independent component analysis. Relative BOLD activations of functional MR-images can be calculated for a time-resolved depiction of hemodynamic changes. The resulting animation can be displayed along with its corresponding paradigm matrix and the presented stimuli. BrainBlend is particularly suitable for the visual exploration of interactions between functional networks, for time-resolved animations of BOLD changes and meets high demands on visual quality in images and animations.

BCI control using 4 direction spatial visual attention and real-time fMRI at 7T

The goal of Brain-Computer-Interface (BCI) technologies is to "outsource" the muscular control to a computer and create new communication channels, e.g. to people with severe paralysis, by measuring cortical activation changes and linking these changes to commands. Using real-time fMRI at 7T we show that visuospatial attention can be used to reliably regulate cortical activity and that it is possible to separate the cortical responses to multiple attention target regions in real time. The activated regions were first located on the fly using an incremental statistical analysis and the subjects were then given feedback based on the activity in these regions. Visuospatial attention is an attractive addition to the existing BCI control strategies, and the fact that it leaves the motor system still available makes it suitable also for applications aimed for healthy people.

Activation detection in diffuse optical imaging by means of the general linear model

Due to its non-invasive nature and low cost, diffuse optical imaging (DOI) is becoming a commonly used technique to assess functional activation in the brain. When imaging with DOI, two major issues arise in the data analysis: (i) the separation of noise of physiological origin and the recovery of the functional response; (ii) the tomographic image reconstruction problem. This paper focuses on the first issue. Although the general linear model (GLM) has been extensively used in functional magnetic resonance imaging (fMRI), DOI has mostly relied on filtering and averaging of raw data to recover brain functional activation. This is mainly due to the high temporal resolution of DOI which implies a new design of the drift basis modelling physiology. In this paper, we provide (i) a filtering method based on cosine functions that is more adapted than standard averaging techniques for DOI specifically; (ii) a new mode-locking technique to recover small signals and locate them temporally with high precision (shift method). Results on real data show the capability of the shift method to retrieve HbR and HbO(2) peak locations.

Real-time functional magnetic resonance imaging: methods and applications

Functional magnetic resonance imaging (fMRI) has been limited by time-consuming data analysis and a low signal-to-noise ratio, impeding online analysis. Recent advances in acquisition techniques, computational power and algorithms increased the sensitivity and speed of fMRI significantly, making real-time analysis and display of fMRI data feasible. So far, most reports have focused on the technical aspects of real-time fMRI (rtfMRI). Here, we provide an overview of the different major areas of applications that became possible with rtfMRI: online analysis of single-subject data provides immediate quality assurance and functional localizers guiding the main fMRI experiment or surgical interventions. In teaching, rtfMRI naturally combines all essential parts of a neuroimaging experiment, such as experimental design, data acquisition and analysis, while adding a high level of interactivity. Thus, the learning of essential knowledge required to conduct functional imaging experiments is facilitated. rtfMRI allows for brain-computer interfaces (BCI) with a high spatial and temporal resolution and whole-brain coverage. Recent studies have shown that such BCI can be used to provide online feedback of the blood-oxygen-level-dependent signal and to learn the self-regulation of local brain activity. Preliminary evidence suggests that this local self-regulation can be used as a new paradigm in cognitive neuroscience to study brain plasticity and the functional relevance of brain areas, even being potentially applicable for psychophysiological treatment.

Real-time functional MRI: development and emerging applications

Real-time functional magnetic resonance imaging (fMRI) is an emerging technique for assessing the dynamic and robust changes in brain activation during an ongoing experiment. Real-time fMRI allows measurement of several processes within the brain as they occur. The extracted information can be used to monitor the quality of acquired data sets, serve as the basis for neurofeedback training, and manipulate scans for interactive paradigm designs. Although more work is needed, recent results have demonstrated a variety of potential applications for real-time fMRI for research and clinical use. We discuss these developments and focus on methods enabling real-time analysis of fMRI data sets, novel research applications arising from these approaches, and potential use of real-time fMRI in clinical settings.

Dynamic monitoring of brain activation under visual stimulation using fMRI—The advantage of real-time fMRI with sliding window GLM analysis

An fMRI technique based on real-time analysis was applied to evaluate the advantages of dynamic monitoring of the t-statistics based on a general linear model. The temporal change of the t-statistics in V1 and V4 under four conditions of visual stimuli covering different visual fields with or without coloring was estimated using an incremental analysis and a sliding window analysis (SWA). The SWA not only visualized the dynamic change of the activation in response to the task conditions and switching, but also enabled us to evaluate the temporal correlation of the t-statistics among the four visual areas. It was suggested that the activity in the V4 was bilaterally organized, and the altering color stimuli gave stronger stimulation to the V1 than did the black and white stimuli. Although the activation map at each time point represents the brain activity during several task and rest blocks, a SWA will be useful to evaluate the transition of neuronal activation in response to several sequential task conditions. An incremental analysis will be useful to monitor the ongoing activation in real-time during the scan, since it gives a higher t-value according to the accumulation of volume data. These two methods will be complementary.