1
|
Pulferer HS, Guan C, Müller-Putz GR. Investigating multilevel cognitive processing within error-free and error-prone feedback conditions in executed and observed car driving. Front Hum Neurosci 2024; 18:1383956. [PMID: 38993330 PMCID: PMC11236611 DOI: 10.3389/fnhum.2024.1383956] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2024] [Accepted: 06/17/2024] [Indexed: 07/13/2024] Open
Abstract
Accident analyses repeatedly reported the considerable contribution of run-off-road incidents to fatalities in road traffic, and despite considerable advances in assistive technologies to mitigate devastating consequences, little insight into the drivers' brain response during such accident scenarios has been gained. While various literature documents neural correlates to steering motion, the driver's mental state, and the impact of distraction and fatigue on driving performance, the cortical substrate of continuous deviations of a car from the road - i.e., how the brain represents a varying discrepancy between the intended and observed car position and subsequently assigns customized levels of corrective measures - remains unclear. Furthermore, the superposition of multiple subprocesses, such as visual and erroneous feedback processing, performance monitoring, or motor control, complicates a clear interpretation of engaged brain regions within car driving tasks. In the present study, we thus attempted to disentangle these subprocesses, employing passive and active steering conditions within both error-free and error-prone vehicle operation conditions. We recorded EEG signals of 26 participants in 13 sessions, simultaneously measuring pairs of Executors (actively steering) and Observers (strictly observing) during a car driving task. We observed common brain patterns in the Executors regardless of error-free or error-prone vehicle operation, albeit with a shift in spectral activity from motor beta to occipital alpha oscillations within erroneous conditions. Further, significant frontocentral differences between Observers and Executors, tracing back to the caudal anterior cingulate cortex, arose during active steering conditions, indicating increased levels of motor-behavioral cognitive control. Finally, we present regression results of both the steering signal and the car position, indicating that a regression of continuous deviations from the road utilizing the EEG might be feasible.
Collapse
Affiliation(s)
- Hannah S. Pulferer
- Institute of Neural Engineering, Graz University of Technology, Graz, Austria
| | - Cuntai Guan
- College of Computing and Data Science, Nanyang Technological University, Singapore, Singapore
| | - Gernot R. Müller-Putz
- Institute of Neural Engineering, Graz University of Technology, Graz, Austria
- BioTechMed-Graz, Graz, Austria
| |
Collapse
|
2
|
Kim JS, Kim H, Chung CK, Kim JS. Dual model transfer learning to compensate for individual variability in brain-computer interface. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 254:108294. [PMID: 38943984 DOI: 10.1016/j.cmpb.2024.108294] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/18/2024] [Revised: 06/14/2024] [Accepted: 06/16/2024] [Indexed: 07/01/2024]
Abstract
BACKGROUND AND OBJECTIVE Recent advancements in brain-computer interface (BCI) technology have seen a significant shift towards incorporating complex decoding models such as deep neural networks (DNNs) to enhance performance. These models are particularly crucial for sophisticated tasks such as regression for decoding arbitrary movements. However, these BCI models trained and tested on individual data often face challenges with limited performance and generalizability across different subjects. This limitation is primarily due to a tremendous number of parameters of DNN models. Training complex models demands extensive datasets. Nevertheless, group data from many subjects may not produce sufficient decoding performance because of inherent variability in neural signals both across individuals and over time METHODS: To address these challenges, this study proposed a transfer learning approach that could effectively adapt to subject-specific variability in cortical regions. Our method involved training two separate movement decoding models: one on individual data and another on pooled group data. We then created a salience map for each cortical region from the individual model, which helped us identify the input's contribution variance across subjects. Based on the contribution variance, we combined individual and group models using a modified knowledge distillation framework. This approach allowed the group model to be universally applicable by assigning greater weights to input data, while the individual model was fine-tuned to focus on areas with significant individual variance RESULTS: Our combined model effectively encapsulated individual variability. We validated this approach with nine subjects performing arm-reaching tasks, with our method outperforming (mean correlation coefficient, r = 0.75) both individual (r = 0.70) and group models (r = 0.40) in decoding performance. In particular, there were notable improvements in cases where individual models showed low performances (e.g., r = 0.50 in the individual decoder to r = 0.61 in the proposed decoder) CONCLUSIONS: These results not only demonstrate the potential of our method for robust BCI, but also underscore its ability to generalize individual data for broader applicability.
Collapse
Affiliation(s)
- Jun Su Kim
- Dept. of Brain and Cognitive Sciences, Seoul National University, Seoul, Republic of Korea; Clinical Research Institute, Konkuk University Medical Center Seoul, Republic of Korea
| | - HongJune Kim
- Clinical Research Institute, Konkuk University Medical Center Seoul, Republic of Korea
| | - Chun Kee Chung
- Dept. of Brain and Cognitive Sciences, Seoul National University, Seoul, Republic of Korea; Interdisciplinary Program in Neuroscience, Seoul National University, Seoul, Republic of Korea; Dept. of Neurosurgery, Seoul National University Hospital, Seoul, Republic of Korea; Neuroscience Research Institute, Seoul National University Medical Research Center, Seoul, Republic of Korea
| | - June Sic Kim
- Clinical Research Institute, Konkuk University Medical Center Seoul, Republic of Korea; Research Institute of Biomedical Science & Technology, Konkuk University, Seoul, Republic of Korea.
| |
Collapse
|
3
|
Jain A, Kumar L. EEG Cortical Source Feature based Hand Kinematics Decoding using Residual CNN-LSTM Neural Network. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023; 2023:1-4. [PMID: 38082886 DOI: 10.1109/embc40787.2023.10341052] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/18/2023]
Abstract
Motor kinematics decoding (MKD) using brain signal is essential to develop Brain-computer interface (BCI) system for rehabilitation or prosthesis devices. Surface electroencephalogram (EEG) signal has been widely utilized for MKD. However, kinematic decoding from cortical sources is sparsely explored. In this work, the feasibility of hand kinematics decoding using EEG cortical source signals has been explored for grasp and lift task. In particular, pre-movement EEG segment is utilized. A residual convolutional neural network (CNN) - long short-term memory (LSTM) based kinematics decoding model is proposed that utilizes motor neural information present in pre-movement brain activity. Various EEG windows at 50 ms prior to movement onset, are utilized for hand kinematics decoding. Correlation value (CV) between actual and predicted hand kinematics is utilized as performance metric for source and sensor domain. The performance of the proposed deep learning model is compared in sensor and source domain. The results demonstrate the viability of hand kinematics decoding using pre-movement EEG cortical source data.
Collapse
|
4
|
Pulferer HS, Kostoglou K, Müller-Putz GR. Getting off track: Cortical feedback processing network modulated by continuous error signal during target-feedback mismatch. Neuroimage 2023; 274:120144. [PMID: 37121373 DOI: 10.1016/j.neuroimage.2023.120144] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/13/2022] [Revised: 04/14/2023] [Accepted: 04/27/2023] [Indexed: 05/02/2023] Open
Abstract
Performance monitoring and feedback processing - especially in the wake of erroneous outcomes - represent a crucial aspect of everyday life, allowing us to deal with imminent threats in the short term but also promoting necessary behavioral adjustments in the long term to avoid future conflicts. Over the last thirty years, research extensively analyzed the neural correlates of processing discrete error stimuli, unveiling the error-related negativity (ERN) and error positivity (Pe) as two main components of the cognitive response. However, the connection between the ERN/Pe and distinct stages of error processing, ranging from action monitoring to subsequent corrective behavior, remains ambiguous. Furthermore, mundane actions such as steering a vehicle already transgress the scope of discrete erroneous events and demand fine-tuned feedback control, and thus, the processing of continuous error signals - a topic scarcely researched at present. We analyzed two electroencephalography datasets to investigate the processing of continuous erroneous signals during a target tracking task, employing feedback in various levels and modalities. We observed significant differences between correct (slightly delayed) and erroneous feedback conditions in the larger one of the two datasets that we analyzed, both in sensor and source space. Furthermore, we found strong error-induced modulations that appeared consistent across datasets and error conditions, indicating a clear order of engagement of specific brain regions that correspond to individual components of error processing.
Collapse
Affiliation(s)
- Hannah S Pulferer
- Institute of Neural Engineering, Graz University of Technology, Stremayrgasse 16/IV, Graz, Austria
| | - Kyriaki Kostoglou
- Institute of Neural Engineering, Graz University of Technology, Stremayrgasse 16/IV, Graz, Austria
| | - Gernot R Müller-Putz
- Institute of Neural Engineering, Graz University of Technology, Stremayrgasse 16/IV, Graz, Austria; BioTechMed-Graz, Graz, Austria.
| |
Collapse
|
5
|
Srisrisawang N, Müller-Putz GR. Transfer Learning in Trajectory Decoding: Sensor or Source Space? SENSORS (BASEL, SWITZERLAND) 2023; 23:3593. [PMID: 37050653 PMCID: PMC10098869 DOI: 10.3390/s23073593] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/19/2023] [Revised: 03/08/2023] [Accepted: 03/28/2023] [Indexed: 06/19/2023]
Abstract
In this study, across-participant and across-session transfer learning was investigated to minimize the calibration time of the brain-computer interface (BCI) system in the context of continuous hand trajectory decoding. We reanalyzed data from a study with 10 able-bodied participants across three sessions. A leave-one-participant-out (LOPO) model was utilized as a starting model. Recursive exponentially weighted partial least squares regression (REW-PLS) was employed to overcome the memory limitation due to the large pool of training data. We considered four scenarios: generalized with no update (Gen), generalized with cumulative update (GenC), and individual models with cumulative (IndC) and non-cumulative (Ind) updates, with each one trained with sensor-space features or source-space features. The decoding performance in generalized models (Gen and GenC) was lower than the chance level. In individual models, the cumulative update (IndC) showed no significant improvement over the non-cumulative model (Ind). The performance showed the decoder's incapability to generalize across participants and sessions in this task. The results suggested that the best correlation could be achieved with the sensor-space individual model, despite additional anatomical information in the source-space features. The decoding pattern showed a more localized pattern around the precuneus over three sessions in Ind models.
Collapse
Affiliation(s)
- Nitikorn Srisrisawang
- Institute of Neural Engineering, Graz University of Technology, Stremayrgasse 16/IV, 8010 Graz, Austria
| | - Gernot R. Müller-Putz
- Institute of Neural Engineering, Graz University of Technology, Stremayrgasse 16/IV, 8010 Graz, Austria
- BioTechMed Graz, 8010 Graz, Austria
| |
Collapse
|
6
|
Borra D, Fantozzi S, Bisi MC, Magosso E. Modulations of Cortical Power and Connectivity in Alpha and Beta Bands during the Preparation of Reaching Movements. SENSORS (BASEL, SWITZERLAND) 2023; 23:3530. [PMID: 37050590 PMCID: PMC10099070 DOI: 10.3390/s23073530] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/26/2023] [Revised: 03/24/2023] [Accepted: 03/25/2023] [Indexed: 06/19/2023]
Abstract
Planning goal-directed movements towards different targets is at the basis of common daily activities (e.g., reaching), involving visual, visuomotor, and sensorimotor brain areas. Alpha (8-13 Hz) and beta (13-30 Hz) oscillations are modulated during movement preparation and are implicated in correct motor functioning. However, how brain regions activate and interact during reaching tasks and how brain rhythms are functionally involved in these interactions is still limitedly explored. Here, alpha and beta brain activity and connectivity during reaching preparation are investigated at EEG-source level, considering a network of task-related cortical areas. Sixty-channel EEG was recorded from 20 healthy participants during a delayed center-out reaching task and projected to the cortex to extract the activity of 8 cortical regions per hemisphere (2 occipital, 2 parietal, 3 peri-central, 1 frontal). Then, we analyzed event-related spectral perturbations and directed connectivity, computed via spectral Granger causality and summarized using graph theory centrality indices (in degree, out degree). Results suggest that alpha and beta oscillations are functionally involved in the preparation of reaching in different ways, with the former mediating the inhibition of the ipsilateral sensorimotor areas and disinhibition of visual areas, and the latter coordinating disinhibition of the contralateral sensorimotor and visuomotor areas.
Collapse
Affiliation(s)
- Davide Borra
- Department of Electrical, Electronic and Information Engineering “Guglielmo Marconi” (DEI), University of Bologna, Cesena Campus, 47521 Cesena, Italy; (D.B.); (M.C.B.); (E.M.)
| | - Silvia Fantozzi
- Department of Electrical, Electronic and Information Engineering “Guglielmo Marconi” (DEI), University of Bologna, Cesena Campus, 47521 Cesena, Italy; (D.B.); (M.C.B.); (E.M.)
- Interdepartmental Center for Industrial Research on Health Sciences & Technologies, University of Bologna, 40064 Bologna, Italy
| | - Maria Cristina Bisi
- Department of Electrical, Electronic and Information Engineering “Guglielmo Marconi” (DEI), University of Bologna, Cesena Campus, 47521 Cesena, Italy; (D.B.); (M.C.B.); (E.M.)
- Interdepartmental Center for Industrial Research on Health Sciences & Technologies, University of Bologna, 40064 Bologna, Italy
| | - Elisa Magosso
- Department of Electrical, Electronic and Information Engineering “Guglielmo Marconi” (DEI), University of Bologna, Cesena Campus, 47521 Cesena, Italy; (D.B.); (M.C.B.); (E.M.)
- Interdepartmental Center for Industrial Research on Health Sciences & Technologies, University of Bologna, 40064 Bologna, Italy
- Alma Mater Research Institute for Human-Centered Artificial Intelligence, University of Bologna, 40121 Bologna, Italy
| |
Collapse
|