1
|
Salsabilian S, Lee C, Margolis D, Najafizadeh L. An LSTM-based adversarial variational autoencoder framework for self-supervised neural decoding of behavioral choices. J Neural Eng 2024; 21:036052. [PMID: 38621379 DOI: 10.1088/1741-2552/ad3eb3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2023] [Accepted: 04/15/2024] [Indexed: 04/17/2024]
Abstract
Objective.This paper presents data-driven solutions to address two challenges in the problem of linking neural data and behavior: (1) unsupervised analysis of behavioral data and automatic label generation from behavioral observations, and (2) extraction of subject-invariant features for the development of generalized neural decoding models.Approach. For behavioral analysis and label generation, an unsupervised method, which employs an autoencoder to transform behavioral data into a cluster-friendly feature space is presented. The model iteratively refines the assigned clusters with soft clustering assignment loss, and gradually improves the learned feature representations. To address subject variability in decoding neural activity, adversarial learning in combination with a long short-term memory-based adversarial variational autoencoder (LSTM-AVAE) model is employed. By using an adversary network to constrain the latent representations, the model captures shared information among subjects' neural activity, making it proper for cross-subject transfer learning.Main results. The proposed approach is evaluated using cortical recordings of Thy1-GCaMP6s transgenic mice obtained via widefield calcium imaging during a motivational licking behavioral experiment. The results show that the proposed model achieves an accuracy of 89.7% in cross-subject neural decoding, outperforming other well-known autoencoder-based feature learning models. These findings suggest that incorporating an adversary network eliminates subject dependency in representations, leading to improved cross-subject transfer learning performance, while also demonstrating the effectiveness of LSTM-based models in capturing the temporal dependencies within neural data.Significance. Results demonstrate the feasibility of the proposed framework in unsupervised clustering and label generation of behavioral data, as well as achieving high accuracy in cross-subject neural decoding, indicating its potentials for relating neural activity to behavior.
Collapse
Affiliation(s)
- Shiva Salsabilian
- Integrated Systems and NeuroImaging Laboratory, Department of Electrical and Computer Engineering, Rutgers University, Piscataway, NJ 08854, United States of America
| | - Christian Lee
- Department of Cell Biology and Neuroscience, Rutgers University, Piscataway, NJ, United States of America
| | - David Margolis
- Department of Cell Biology and Neuroscience, Rutgers University, Piscataway, NJ, United States of America
| | - Laleh Najafizadeh
- Integrated Systems and NeuroImaging Laboratory, Department of Electrical and Computer Engineering, Rutgers University, Piscataway, NJ 08854, United States of America
| |
Collapse
|
2
|
Subramanian A, Najafizadeh L. Hierarchical Classification Strategy for Mitigating the Impact of The Presence of Pain in fNIRS-based BCIs. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023; 2023:1-4. [PMID: 38083794 DOI: 10.1109/embc40787.2023.10341152] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/18/2023]
Abstract
Brain computer interfaces (BCIs) can find applications in assistive systems for patients who experience conditions that impede their motor abilities. A BCI uses signals acquired from the brain to control external devices. As physical pain influences cortical signals, the presence of pain can negatively impact the performance of the BCI. In this work, we propose a strategy to mitigate this negative impact. Cortical signals are acquired from test subjects while they performed two mental arithmetic tasks, in the presence and the absence of painful stimuli. The task of the BCI is to reliably classify the two mental arithmetic tasks from the cortical recordings, irrespective of the presence or the absence of pain. We propose to do this classification, hierarchically, in two levels. In the first level, the data is classified into those captured in the presence and the absence of pain. Depending on the results of the classification from the first level, in the second level, the BCI performs the classification of tasks using a classifier trained either in the presence or the absence of pain. A 1-dimensional convolutional neural network (1D-CNN) is used for classification at both levels. It is observed that using this hierarchical strategy, the BCI is able to classify the tasks with an accuracy greater than 90%, irrespective of the presence or the absence of pain. Given that the presence of physical pain has shown previously to reduce the classification accuracy of a BCI to almost chance levels, this mitigation strategy will be a significant step towards enhancing the performance of BCIs when they are used in assistive systems for patients.
Collapse
|
3
|
Campbell O, Vanderwal T, Weber AM. Fractal-Based Analysis of fMRI BOLD Signal During Naturalistic Viewing Conditions. Front Physiol 2022; 12:809943. [PMID: 35087421 PMCID: PMC8787275 DOI: 10.3389/fphys.2021.809943] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2021] [Accepted: 12/14/2021] [Indexed: 01/04/2023] Open
Abstract
Background: Temporal fractals are characterized by prominent scale-invariance and self-similarity across time scales. Monofractal analysis quantifies this scaling behavior in a single parameter, the Hurst exponent (H). Higher H reflects greater correlation in the signal structure, which is taken as being more fractal. Previous fMRI studies have observed lower H during conventional tasks relative to resting state conditions, and shown that H is negatively correlated with task difficulty and novelty. To date, no study has investigated the fractal dynamics of BOLD signal during naturalistic conditions. Methods: We performed fractal analysis on Human Connectome Project 7T fMRI data (n = 72, 41 females, mean age 29.46 ± 3.76 years) to compare H across movie-watching and rest. Results: In contrast to previous work using conventional tasks, we found higher H values for movie relative to rest (mean difference = 0.014; p = 5.279 × 10-7; 95% CI [0.009, 0.019]). H was significantly higher in movie than rest in the visual, somatomotor and dorsal attention networks, but was significantly lower during movie in the frontoparietal and default networks. We found no cross-condition differences in test-retest reliability of H. Finally, we found that H of movie-derived stimulus properties (e.g., luminance changes) were fractal whereas H of head motion estimates were non-fractal. Conclusions: Overall, our findings suggest that movie-watching induces fractal signal dynamics. In line with recent work characterizing connectivity-based brain state dynamics during movie-watching, we speculate that these fractal dynamics reflect the configuring and reconfiguring of brain states that occurs during naturalistic processing, and are markedly different than dynamics observed during conventional tasks.
Collapse
Affiliation(s)
- Olivia Campbell
- School of Biomedical Engineering, University of British Columbia, Vancouver, BC, Canada
| | - Tamara Vanderwal
- British Columbia (BC) Children's Hospital Research Institute, UBC, Vancouver, BC, Canada.,Department of Psychiatry, University of British Columbia, Vancouver, BC, Canada
| | - Alexander Mark Weber
- School of Biomedical Engineering, University of British Columbia, Vancouver, BC, Canada.,British Columbia (BC) Children's Hospital Research Institute, UBC, Vancouver, BC, Canada.,Division of Neurology, Department of Pediatrics, University of British Columbia, Vancouver, BC, Canada.,Department of Neuroscience, University of British Columbia, Vancouver, BC, Canada
| |
Collapse
|
4
|
Hu XS, Nascimento TD, DaSilva AF. Shedding light on pain for the clinic: a comprehensive review of using functional near-infrared spectroscopy to monitor its process in the brain. Pain 2021; 162:2805-2820. [PMID: 33990114 PMCID: PMC8490487 DOI: 10.1097/j.pain.0000000000002293] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2020] [Accepted: 03/29/2021] [Indexed: 11/27/2022]
Abstract
ABSTRACT Pain is a complex experience that involves sensation, emotion, and cognition. The subjectivity of the traditional pain measurement tools has expedited the interest in developing neuroimaging techniques to monitor pain objectively. Among noninvasive neuroimaging techniques, functional near-infrared spectroscopy (fNIRS) has balanced spatial and temporal resolution; yet, it is portable, quiet, and cost-effective. These features enable fNIRS to image the cortical mechanisms of pain in a clinical environment. In this article, we evaluated pain neuroimaging studies that used the fNIRS technique in the past decade. Starting from the experimental design, we reviewed the regions of interest, probe localization, data processing, and primary findings of these existing fNIRS studies. We also discussed the fNIRS imaging's potential as a brain surveillance technique for pain, in combination with artificial intelligence and extended reality techniques. We concluded that fNIRS is a brain imaging technique with great potential for objective pain assessment in the clinical environment.
Collapse
Affiliation(s)
- Xiao-Su Hu
- University of Michigan, School of Dentistry, Biologic & Materials Sciences Department, Hedache & Orofacial Pain Effort Lab
| | - Thiago D. Nascimento
- University of Michigan, School of Dentistry, Biologic & Materials Sciences Department, Hedache & Orofacial Pain Effort Lab
| | - Alexandre F. DaSilva
- University of Michigan, School of Dentistry, Biologic & Materials Sciences Department, Hedache & Orofacial Pain Effort Lab
| |
Collapse
|
5
|
Wang W, Mohseni P, Kilgore KL, Najafizadeh L. Cuff-less Blood Pressure Estimation from Photoplethysmography via Visibility Graph and Transfer Learning. IEEE J Biomed Health Inform 2021; 26:2075-2085. [PMID: 34784289 DOI: 10.1109/jbhi.2021.3128383] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
This paper presents a new solution that enables the use of transfer learning for cuff-less blood pressure (BP) monitoring via short duration of photoplethysmogram (PPG). The proposed method estimates BP with low computational budget by 1) creating images from segments of PPG via visibility graph (VG) that preserves the temporal information of the PPG waveform, 2) using pre-trained deep convolutional neural network (CNN) to extract feature vectors from VG images, and 3) solving for the weights and bias between the feature vectors and the reference BPs with ridge regression. Using the University of California Irvine (UCI) database consisting of 348 records, the proposed method achieves a best error performance of 0.008.46 mmHg for systolic blood pressure (SBP), and -0.045.36 mmHg for diastolic blood pressure (DBP), respectively, in terms of the mean error (ME) and the standard deviation (SD) of error, ranking grade B for SBP and grade A for DBP under the British Hypertension Society (BHS) protocol. Our novel data-driven method offers a computationally-efficient end-to-end solution for rapid and user-friendly cuff-less PPG-based BP estimation.
Collapse
|
6
|
Wang W, Mohseni P, Kilgore K, Najafizadeh L. Cuff-Less Blood Pressure Estimation via Small Convolutional Neural Networks. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021; 2021:1031-1034. [PMID: 34891464 DOI: 10.1109/embc46164.2021.9630557] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Deep learning-based cuff-less blood pressure (BP) estimation methods have recently gained increased attention as they can provide accurate BP estimation with only one physiological signal as input. In this paper, we present a simple and effective method for cuff-less BP estimation by training a small-scale convolutional neural network (CNN), modified from LeNet-5, with images created from short segments of the photoplethysmogram (PPG) signal via visibility graph (VG). Results show that the trained modified LeNet-5 model achieves an error performance of 0.184±7.457 mmHg for the systolic BP (SBP), and 0.343±4.065 mmHg for the diastolic BP (DBP) in terms of the mean error (ME) and the standard deviation (SD) of error between the estimated and reference BP. Both the SBP and the DBP accuracy rank grade A under the British Hypertension Society (BHS) protocol, demonstrating that our proposed method is an accurate way for cuff-less BP estimation.
Collapse
|
7
|
Kaszas A, Szalay G, Slézia A, Bojdán A, Vanzetta I, Hangya B, Rózsa B, O'Connor R, Moreau D. Two-photon GCaMP6f imaging of infrared neural stimulation evoked calcium signals in mouse cortical neurons in vivo. Sci Rep 2021; 11:9775. [PMID: 33963220 PMCID: PMC8105372 DOI: 10.1038/s41598-021-89163-x] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2020] [Accepted: 04/22/2021] [Indexed: 02/07/2023] Open
Abstract
Infrared neural stimulation is a promising tool for stimulating the brain because it can be used to excite with high spatial precision without the need of delivering or inserting any exogenous agent into the tissue. Very few studies have explored its use in the brain, as most investigations have focused on sensory or motor nerve stimulation. Using intravital calcium imaging with the genetically encoded calcium indicator GCaMP6f, here we show that the application of infrared neural stimulation induces intracellular calcium signals in Layer 2/3 neurons in mouse cortex in vivo. The number of neurons exhibiting infrared-induced calcium response as well as the amplitude of those signals are shown to be both increasing with the energy density applied. By studying as well the spatial extent of the stimulation, we show that reproducibility of the stimulation is achieved mainly in the central part of the infrared beam path. Stimulating in vivo at such a degree of precision and without any exogenous chromophores enables multiple applications, from mapping the brain's connectome to applications in systems neuroscience and the development of new therapeutic tools for investigating the pathological brain.
Collapse
Affiliation(s)
- Attila Kaszas
- Mines Saint-Etienne, Centre CMP, Département BEL, F - 13541, Gardanne, France
- Institut de Neurosciences de la Timone, CNRS UMR 7289 & Aix-Marseille Université, 13005, Marseille, France
| | - Gergely Szalay
- Laboratory of 3D Functional Network and Dendritic Imaging, Institute of Experimental Medicine, Budapest, 1083, Hungary
| | - Andrea Slézia
- Mines Saint-Etienne, Centre CMP, Département BEL, F - 13541, Gardanne, France
- Lendület Laboratory of Systems Neuroscience, Institute of Experimental Medicine, Budapest, 1083, Hungary
| | - Alexandra Bojdán
- Laboratory of 3D Functional Network and Dendritic Imaging, Institute of Experimental Medicine, Budapest, 1083, Hungary
| | - Ivo Vanzetta
- Institut de Neurosciences de la Timone, CNRS UMR 7289 & Aix-Marseille Université, 13005, Marseille, France
| | - Balázs Hangya
- Lendület Laboratory of Systems Neuroscience, Institute of Experimental Medicine, Budapest, 1083, Hungary
| | - Balázs Rózsa
- Laboratory of 3D Functional Network and Dendritic Imaging, Institute of Experimental Medicine, Budapest, 1083, Hungary
- Two-Photon Laboratory, Faculty of Information Technology, Pázmány Péter Catholic University, Budapest, 1083, Hungary
| | - Rodney O'Connor
- Mines Saint-Etienne, Centre CMP, Département BEL, F - 13541, Gardanne, France
- Institut de Neurosciences de la Timone, CNRS UMR 7289 & Aix-Marseille Université, 13005, Marseille, France
| | - David Moreau
- Mines Saint-Etienne, Centre CMP, Département BEL, F - 13541, Gardanne, France.
| |
Collapse
|