Multi-source domain adaptation for decoder calibration of intracortical brain-machine interface

Overcoming the Domain Gap in Neural Action Representations

Relating behavior to brain activity in animals is a fundamental goal in neuroscience, with practical applications in building robust brain-machine interfaces. However, the domain gap between individuals is a major issue that prevents the training of general models that work on unlabeled subjects. Since 3D pose data can now be reliably extracted from multi-view video sequences without manual intervention, we propose to use it to guide the encoding of neural action representations together with a set of neural and behavioral augmentations exploiting the properties of microscopy imaging. To test our method, we collect a large dataset that features flies and their neural activity. To reduce the domain gap, during training, we mix features of neural and behavioral data across flies that seem to be performing similar actions. To show our method can generalize further neural modalities and other downstream tasks, we test our method on a human neural Electrocorticography dataset, and another RGB video data of human activities from different viewpoints. We believe our work will enable more robust neural decoding algorithms to be used in future brain-machine interfaces.

A behavioral paradigm for cortical control of a robotic actuator by freely moving rats in a one-dimensional two-target reaching task

BACKGROUND

Controlling the trajectory of a neuroprosthesis to reach distant targets is a commonly used brain-machine interface (BMI) task in primates and has not been available for rodents yet.

NEW METHOD

Here, we describe a novel, fine-tuned behavioral paradigm and setup which enables this task for rats in one-dimensional space for reaching two distant targets depending on their limited cognitive and visual capabilities compared to those of primates. An online transform was used to convert the activity of a pair of primary motor cortex (M1) units into two robotic actions. The rats were shaped to adapt to the transform and direct the robotic actuator toward the selected target by modulating the activity of the M1 neurons.

RESULTS

All three rats involved in the study were capable of achieving randomly selected targets with at least 78% accuracy. A total of 9 out of 16 pairs of units examined were eligible for exceeding this success criterion. Two out of three rats were capable of reversal learning, where the mapping between the activity of the M1 units and the robotic actions were reversed.

COMPARISON WITH EXISTING METHODS

The present work is the first demonstration of trajectory-based control of a neuroprosthetic device by rodents to reach two distant targets using visual feedback.

CONCLUSION

The behavioral paradigm and setup introduced here can be used as a cost-effective platform for elucidating the information processing principles in the neural circuits related to neuroprosthetic control and for studying the performance of novel BMI technologies using freely moving rats.

P3-MSDA: Multi-Source Domain Adaptation Network for Dynamic Visual Target Detection

Single-trial electroencephalogram detection has been widely applied in brain-computer interface (BCI) systems. Moreover, an individual generalized model is significant for applying the dynamic visual target detection BCI system in real life because of the time jitter of the detection latency, the dynamics and complexity of visual background. Hence, we developed an unsupervised multi-source domain adaptation network (P3-MSDA) for dynamic visual target detection. In this network, a P3 map-clustering method was proposed for source domain selection. The adversarial domain adaptation was conducted for domain alignment to eliminate individual differences, and prediction probabilities were ranked and returned to guide the input of target samples for imbalanced data classification. The results showed that individuals with a strong P3 map selected by the proposed P3 map-clustering method perform best on the source domain. Compared with existing schemes, the proposed P3-MSDA network achieved the highest classification accuracy and F1 score using five labeled individuals with a strong P3 map as the source domain. These findings can have a significant meaning in building an individual generalized model for dynamic visual target detection.

A behavioral paradigm for cortical control of a robotic actuator by freely moving rats in a one-dimensional two-target reaching task.. [DOI: 10.1101/2021.01.24.427325] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

Controlling the trajectory of a neuroprosthetic device to reach multiple targets is a commonly used brain-machine interface (BMI) task in primates and has not been available for rodents yet. Here, we describe a novel experimental paradigm which enables this task for rats in one-dimensional space for reaching two distant targets depending on their limited cognitive and visual capabilities compared to primates. An online transform was used to convert the activity of a pair of primary motor cortex (M1) units into two robotic actions. The rats were shaped to adapt to the transform and direct the robotic actuator toward the selected target by modulating the activity of the M1 neurons. All three rats involved in the study were capable of achieving randomly selected targets with at least 78% accuracy. A total of 9 out of 16 pairs of units examined were eligible for exceeding this success criterion. Two out of three rats were capable of reversal learning, where the mapping between the activity of the unit pairs and the robotic actions were reversed. The present work is the first demonstration of trajectory-based control of a neuroprosthetic device by rodents to reach two distant targets using visual feedback. The paradigm introduced here may be used as a cost-effective platform for elucidating the information processing principles in the neural circuits related to neuroprosthetic control and for studying the performance of novel BMI technologies using freely moving rats.