Sense of agency for intracortical brain-machine interfaces

The Body Knows Better: Sensorimotor signals reveal the interplay between implicit and explicit Sense of Agency in the human mind

Sense of Agency (SoA) is the feeling of control over our actions. SoA has been suggested to arise from both implicit sensorimotor integration as well as higher-level decision processes. SoA is typically measured by collecting participants' subjective judgments, conflating both implicit and explicit processing. Consequently, the interplay between implicit sensorimotor processing and explicit agency judgments is not well understood. Here, we evaluated in one exploratory and one preregistered experiment (N = 60), using a machine learning approach, the relation between a well-known mechanism of implicit sensorimotor adaptation and explicit SoA judgments. Specifically, we examined whether subjective judgments of SoA and sensorimotor conflicts could be inferred from hand kinematics in a sensorimotor task using a virtual hand (VH). In both experiments participants performed a hand movement and viewed a virtual hand making a movement that could either be synchronous with their action or include a parametric temporal delay. After each movement, participants judged whether their actual movement was congruent with the movement they observed. Our results demonstrated that sensorimotor conflicts could be inferred from implicit motor kinematics on a trial by trial basis. Moreover, detection of sensorimotor conflicts from machine learning models of kinematic data provided more accurate classification of sensorimotor congruence than participants' explicit judgments. These results were replicated in a second, preregistered, experiment. These findings show evidence of diverging implicit and explicit processing for SoA and suggest that the brain holds high-quality information on sensorimotor conflicts that is not fully utilized in the inference of conscious agency.

Influence of feedback transparency on motor imagery neurofeedback performance: the contribution of agency

Objective.Neurofeedback (NF) is a cognitive training procedure based on real-time feedback (FB) of a participant's brain activity that they must learn to self-regulate. A classical visual FB delivered in a NF task is a filling gauge reflecting a measure of brain activity. This abstract visual FB is not transparently linked-from the subject's perspective-to the task performed (e.g., motor imagery (MI)). This may decrease the sense of agency, that is, the participants' reported control over FB. Here, we assessed the influence of FB transparency on NF performance and the role of agency in this relationship.Approach.Participants performed a NF task using MI to regulate brain activity measured using electroencephalography. In separate blocks, participants experienced three different conditions designed to vary transparency: FB was presented as either (1) a swinging pendulum, (2) a clenching virtual hand, (3) a clenching virtual hand combined with a motor illusion induced by tendon vibration. We measured self-reported agency and user experience after each NF block.Main results. We found that FB transparency influences NF performance. Transparent visual FB provided by the virtual hand resulted in significantly better NF performance than the abstract FB of the pendulum. Surprisingly, adding a motor illusion to the virtual hand significantly decreased performance relative to the virtual hand alone. When introduced in incremental linear mixed effect models, self-reported agency was significantly associated with NF performance and it captured the variance related to the effect of FB transparency on NF performance.Significance. Our results highlight the relevance of transparent FB in relation to the sense of agency. This is likely an important consideration in designing FB to improve NF performance and learning outcomes.

Sensory integration for neuroprostheses: from functional benefits to neural correlates

In the field of sensory neuroprostheses, one ultimate goal is for individuals to perceive artificial somatosensory information and use the prosthesis with high complexity that resembles an intact system. To this end, research has shown that stimulation-elicited somatosensory information improves prosthesis perception and task performance. While studies strive to achieve sensory integration, a crucial phenomenon that entails naturalistic interaction with the environment, this topic has not been commensurately reviewed. Therefore, here we present a perspective for understanding sensory integration in neuroprostheses. First, we review the engineering aspects and functional outcomes in sensory neuroprosthesis studies. In this context, we summarize studies that have suggested sensory integration. We focus on how they have used stimulation-elicited percepts to maximize and improve the reliability of somatosensory information. Next, we review studies that have suggested multisensory integration. These works have demonstrated that congruent and simultaneous multisensory inputs provided cognitive benefits such that an individual experiences a greater sense of authority over prosthesis movements (i.e., agency) and perceives the prosthesis as part of their own (i.e., ownership). Thereafter, we present the theoretical and neuroscience framework of sensory integration. We investigate how behavioral models and neural recordings have been applied in the context of sensory integration. Sensory integration models developed from intact-limb individuals have led the way to sensory neuroprosthesis studies to demonstrate multisensory integration. Neural recordings have been used to show how multisensory inputs are processed across cortical areas. Lastly, we discuss some ongoing research and challenges in achieving and understanding sensory integration in sensory neuroprostheses. Resolving these challenges would help to develop future strategies to improve the sensory feedback of a neuroprosthetic system.

How different immersive environments affect intracortical brain computer interfaces

As brain-computer interface (BCI) research advances, many new applications are being developed. Tasks can be performed in different environments, and whether a BCI user can switch environments seamlessly will influence the ultimate utility of a clinical device. Here we investigate the importance of the immersiveness of the virtual environment used to train BCI decoders on the resulting decoder and its generalizability between environments. Two participants who had intracortical electrodes implanted in their precentral gyrus used a BCI to control a virtual arm, either viewed immersively through virtual reality goggles or at a distance on a flat television monitor. Each participant performed better with a decoder trained and tested in the environment they had used the most prior to the study, one for each environment type. The neural tuning to the desired movement was minimally influenced by the immersiveness of the environment. Finally, in further testing with one of the participants, we found that decoders trained in one environment generalized well to the other environment, but the order in which the environments were experienced within a session mattered. Overall, experience with an environment was more influential on performance than the immersiveness of the environment, but BCI performance generalized well after accounting for experience.

Intentional binding for noninvasive BCI control

Objective. Noninvasive brain-computer interfaces (BCIs) allow to interact with the external environment by naturally bypassing the musculoskeletal system. Making BCIs efficient and accurate is paramount to improve the reliability of real-life and clinical applications, from open-loop device control to closed-loop neurorehabilitation.Approach. By promoting sense of agency and embodiment, realistic setups including multimodal channels of communication, such as eye-gaze, and robotic prostheses aim to improve BCI performance. However, how the mental imagery command should be integrated in those hybrid systems so as to ensure the best interaction is still poorly understood. To address this question, we performed a hybrid EEG-based BCI training involving healthy volunteers enrolled in a reach-and-grasp action operated by a robotic arm.Main results. Showed that the hand grasping motor imagery timing significantly affects the BCI accuracy evolution as well as the spatiotemporal brain dynamics. Larger accuracy improvement was obtained when motor imagery is performed just after the robot reaching, as compared to before or during the movement. The proximity with the subsequent robot grasping favored intentional binding, led to stronger motor-related brain activity, and primed the ability of sensorimotor areas to integrate information from regions implicated in higher-order cognitive functions.Significance. Taken together, these findings provided fresh evidence about the effects of intentional binding on human behavior and cortical network dynamics that can be exploited to design a new generation of efficient brain-machine interfaces.

Invasive Brain Computer Interface for Motor Restoration in Spinal Cord Injury: A Systematic Review

STUDY DESIGN

Systematic review of the literature.

OBJECTIVES

In recent years, brain-computer interface (BCI) has emerged as a potential treatment for patients with spinal cord injury (SCI). This is the first systematic review of the literature on invasive closed-loop BCI technologies for the treatment of SCI in humans.

MATERIALS AND METHODS

A comprehensive search of PubMed MEDLINE, Web of Science, and Ovid EMBASE was conducted following Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines.

RESULTS

Of 8316 articles collected, 19 studies met all the inclusion criteria. Data from 21 patients were extracted from these studies. All patients sustained a cervical SCI and were treated using either a BCI with intracortical microelectrode arrays (n = 18, 85.7%) or electrocorticography (n = 3, 14.3%). To decode these neural signals, machine learning and statistical models were used: support vector machine in eight patients (38.1%), linear estimator in seven patients (33.3%), Hidden Markov Model in three patients (14.3%), and other in three patients (14.3%). As the outputs, ten patients (47.6%) underwent noninvasive functional electrical stimulation (FES) with a cuff; one (4.8%) had an invasive FES with percutaneous stimulation, and ten (47.6%) used an external device (neuroprosthesis or virtual avatar). Motor function was restored in all patients for each assigned task. Clinical outcome measures were heterogeneous across all studies.

CONCLUSIONS

Invasive techniques of BCI show promise for the treatment of SCI, but there is currently no technology that can restore complete functional autonomy in patients with SCI. The current techniques and outcomes of BCI vary greatly. Because invasive BCIs are still in the early stages of development, further clinical studies should be conducted to optimize the prognosis for patients with SCI.

Self-Improvising Memory: A Perspective on Memories as Agential, Dynamically Reinterpreting Cognitive Glue

Many studies on memory emphasize the material substrate and mechanisms by which data can be stored and reliably read out. Here, I focus on complementary aspects: the need for agents to dynamically reinterpret and modify memories to suit their ever-changing selves and environment. Using examples from developmental biology, evolution, and synthetic bioengineering, in addition to neuroscience, I propose that a perspective on memory as preserving salience, not fidelity, is applicable to many phenomena on scales from cells to societies. Continuous commitment to creative, adaptive confabulation, from the molecular to the behavioral levels, is the answer to the persistence paradox as it applies to individuals and whole lineages. I also speculate that a substrate-independent, processual view of life and mind suggests that memories, as patterns in the excitable medium of cognitive systems, could be seen as active agents in the sense-making process. I explore a view of life as a diverse set of embodied perspectives-nested agents who interpret each other's and their own past messages and actions as best as they can (polycomputation). This synthesis suggests unifying symmetries across scales and disciplines, which is of relevance to research programs in Diverse Intelligence and the engineering of novel embodied minds.

Reconnecting the Hand and Arm to the Brain: Efficacy of Neural Interfaces for Sensorimotor Restoration After Tetraplegia

BACKGROUND AND OBJECTIVES

Paralysis after spinal cord injury involves damage to pathways that connect neurons in the brain to peripheral nerves in the limbs. Re-establishing this communication using neural interfaces has the potential to bridge the gap and restore upper extremity function to people with high tetraplegia. We report a novel approach for restoring upper extremity function using selective peripheral nerve stimulation controlled by intracortical microelectrode recordings from sensorimotor networks, along with restoration of tactile sensation of the hand using intracortical microstimulation.

METHODS

A 27-year-old right-handed man with AIS-B (motor-complete, sensory-incomplete) C3-C4 tetraplegia was enrolled into the clinical trial. Six 64-channel intracortical microelectrode arrays were implanted into left hemisphere regions involved in upper extremity function, including primary motor and sensory cortices, inferior frontal gyrus, and anterior intraparietal area. Nine 16-channel extraneural peripheral nerve electrodes were implanted to allow targeted stimulation of right median, ulnar (2), radial, axillary, musculocutaneous, suprascapular, lateral pectoral, and long thoracic nerves, to produce selective muscle contractions on demand. Proof-of-concept studies were performed to demonstrate feasibility of using a brain-machine interface to read from and write to the brain for restoring motor and sensory functions of the participant's own arm and hand.

RESULTS

Multiunit neural activity that correlated with intended motor action was successfully recorded from intracortical arrays. Microstimulation of electrodes in somatosensory cortex produced repeatable sensory percepts of individual fingers for restoration of touch sensation. Selective electrical activation of peripheral nerves produced antigravity muscle contractions, resulting in functional movements that the participant was able to command under brain control to perform virtual and actual arm and hand movements. The system was well tolerated with no operative complications.

CONCLUSION

The combination of implanted cortical electrodes and nerve cuff electrodes has the potential to create bidirectional restoration of motor and sensory functions of the arm and hand after neurological injury.

A novel theta-controlled vibrotactile brain-computer interface to treat chronic pain: a pilot study

Limitations in chronic pain therapies necessitate novel interventions that are effective, accessible, and safe. Brain-computer interfaces (BCIs) provide a promising modality for targeting neuropathology underlying chronic pain by converting recorded neural activity into perceivable outputs. Recent evidence suggests that increased frontal theta power (4-7 Hz) reflects pain relief from chronic and acute pain. Further studies have suggested that vibrotactile stimulation decreases pain intensity in experimental and clinical models. This longitudinal, non-randomized, open-label pilot study's objective was to reinforce frontal theta activity in six patients with chronic upper extremity pain using a novel vibrotactile neurofeedback BCI system. Patients increased their BCI performance, reflecting thought-driven control of neurofeedback, and showed a significant decrease in pain severity (1.29 ± 0.25 MAD, p = 0.03, q = 0.05) and pain interference (1.79 ± 1.10 MAD p = 0.03, q = 0.05) scores without any adverse events. Pain relief significantly correlated with frontal theta modulation. These findings highlight the potential of BCI-mediated cortico-sensory coupling of frontal theta with vibrotactile stimulation for alleviating chronic pain.

Human primary motor cortex indexes the onset of subjective intention in brain-machine-interface mediated actions

Self-initiated behavior is accompanied by the experience of willing our actions. Here, we leverage the unique opportunity to examine the full intentional chain - from will (W) to action (A) to environmental effects (E) - in a tetraplegic person fitted with a primary motor cortex (M1) brain machine interface (BMI) generating hand movements via neuromuscular electrical stimulation (NMES). This combined BMI-NMES approach allowed us to selectively manipulate each element of the intentional chain (W, A, and E) while performing extra-cellular recordings and probing subjective experience. Our results reveal single-cell, multi-unit, and population-level dynamics in human M1 that encode W and may predict its subjective onset. Further, we show that the proficiency of a neural decoder in M1 reflects the degree of W-A binding, tracking the participant's subjective experience of intention in (near) real time. These results point to M1 as a critical node in forming the subjective experience of intention and demonstrate the relevance of intention-related signals for translational neuroprosthetics.

Synchronous motor imagery and visual feedback of finger movement elicit the moving rubber hand illusion, at least in illusion-susceptible individuals

Recent evidence suggests that imagined auditory and visual sensory stimuli can be integrated with real sensory information from a different sensory modality to change the perception of external events via cross-modal multisensory integration mechanisms. Here, we explored whether imagined voluntary movements can integrate visual and proprioceptive cues to change how we perceive our own limbs in space. Participants viewed a robotic hand wearing a glove repetitively moving its right index finger up and down at a frequency of 1 Hz, while they imagined executing the corresponding movements synchronously or asynchronously (kinesthetic-motor imagery); electromyography (EMG) from the participants' right index flexor muscle confirmed that the participants kept their hand relaxed while imagining the movements. The questionnaire results revealed that the synchronously imagined movements elicited illusory ownership and a sense of agency over the moving robotic hand-the moving rubber hand illusion-compared with asynchronously imagined movements; individuals who affirmed experiencing the illusion with real synchronous movement also did so with synchronous imagined movements. The results from a proprioceptive drift task further demonstrated a shift in the perceived location of the participants' real hand toward the robotic hand in the synchronous versus the asynchronous motor imagery condition. These results suggest that kinesthetic motor imagery can be used to replace veridical congruent somatosensory feedback from a moving finger in the moving rubber hand illusion to trigger illusory body ownership and agency, but only if the temporal congruence rule of the illusion is obeyed. This observation extends previous studies on the integration of mental imagery and sensory perception to the case of multisensory bodily awareness, which has potentially important implications for research into embodiment of brain-computer interface controlled robotic prostheses and computer-generated limbs in virtual reality.

Bioactive polymer-enabled conformal neural interface and its application strategies

Neural interface is a powerful tool to control the varying neuron activities in the brain, where the performance can directly affect the quality of recording neural signals and the reliability of in vivo connection between the brain and external equipment. Recent advances in bioelectronic innovation have provided promising pathways to fabricate flexible electrodes by integrating electrodes on bioactive polymer substrates. These bioactive polymer-based electrodes can enable the conformal contact with irregular tissue and result in low inflammation when compared to conventional rigid inorganic electrodes. In this review, we focus on the use of silk fibroin and cellulose biopolymers as well as certain synthetic polymers to offer the desired flexibility for constructing electrode substrates for a conformal neural interface. First, the development of a neural interface is reviewed, and the signal recording methods and tissue response features of the implanted electrodes are discussed in terms of biocompatibility and flexibility of corresponding neural interfaces. Following this, the material selection, structure design and integration of conformal neural interfaces accompanied by their effective applications are described. Finally, we offer our perspectives on the evolution of desired bioactive polymer-enabled neural interfaces, regarding the biocompatibility, electrical properties and mechanical softness.

Electron Conductive and Transparent Hydrogels for Recording Brain Neural Signals and Neuromodulation

Recording brain neural signals and optogenetic neuromodulations open frontiers in decoding brain neural information and neurodegenerative disease therapeutics. Conventional implantable probes suffer from modulus mismatch with biological tissues and an irreconcilable tradeoff between transparency and electron conductivity. Herein, a strategy is proposed to address these tradeoffs, which generates conductive and transparent hydrogels with polypyrrole-decorated microgels as cross-linkers. The optical transparency of the electrodes can be attributed to the special structures that allow light waves to bypass the microgel particles and minimize their interaction. Demonstrated by probing the hippocampus of rat brains, the biomimetic electrode shows a prolonged capacity for simultaneous optogenetic neuromodulation and recording of brain neural signals. More importantly, an intriguing brain-machine interaction is realized, which involves signal input to the brain, brain neural signal generation, and controlling limb behaviors. This breakthrough work represents a significant scientific advancement toward decoding brain neural information and developing neurodegenerative disease therapies.

Breathing is coupled with voluntary initiation of mental imagery

Previous research has suggested that bodily signals from internal organs are associated with diverse cortical and subcortical processes involved in sensory-motor functions, beyond homeostatic reflexes. For instance, a recent study demonstrated that the preparation and execution of voluntary actions, as well as its underlying neural activity, are coupled with the breathing cycle. In the current study, we investigated whether such breathing-action coupling is limited to voluntary motor action or whether it is also present for mental actions not involving any overt bodily movement. To answer this question, we recorded electroencephalography (EEG), electromyography (EMG), and respiratory signals while participants were conducting a voluntary action paradigm including self-initiated motor execution (ME), motor imagery (MI), and visual imagery (VI) tasks. We observed that the voluntary initiation of ME, MI, and VI are similarly coupled with the respiration phase. In addition, EEG analysis revealed the existence of readiness potential (RP) waveforms in all three tasks (i.e., ME, MI, VI), as well as a coupling between the RP amplitude and the respiratory phase. Our findings show that the voluntary initiation of both imagined and overt action is coupled with respiration, and further suggest that the breathing system is involved in preparatory processes of voluntary action by contributing to the temporal decision of when to initiate the action plan, regardless of whether this culminates in overt movements.

Stretchable Temperature-Responsive Multimodal Neuromorphic Electronic Skin with Spontaneous Synaptic Plasticity Recovery

Multimodal electronic skin devices capable of detecting multimodal signals provide the possibility for health monitoring. Sensing and memory for temperature and deformation by human skin are of great significance for the perception and monitoring of physiological changes of the human body. Electronic skin is highly expected to have similar functions as human skin. Here, by implementing intrinsically stretchable neuromorphic transistors with mechanoreceptors and thermoreceptors in an array, we have realized stretchable temperature-responsive multimodal neuromorphic electronic skin (STRM-NES) with both sensory and memory functions, in which synaptic plasticity can be modulated by multiple modalities, in situ temperature variations, and stretching deformations. Temperature-responsive functions, spontaneous recovery, and temperature-dependent multitrial learning are proposed. Furthermore, a stretchable temperature neuromorphic array composed of multiple fully functional subcells is demonstrated to identify temperature distributions and variations at different regions and conditions after various strains of skin. The STRM-NES has temperature- and strain-responsive neuromorphic functions, excellent self-healing, and reusable capability, showing similar abilities as human skin to sense, transmit, memory, and recovery from external stimuli. It is expected to facilitate the development of wearable electronics, intelligent robotics, and prosthetic applications.

Multisensory Integration in Bionics: Relevance and Perspectives

Purpose of review

The goal of the review is to highlight the growing importance of multisensory integration processes connected to bionic limbs and somatosensory feedback restoration.

Recent findings

Restoring quasi-realistic sensations by means of neurostimulation has been shown to provide functional and motor benefits in limb amputees. In the recent past, cognitive processes linked to the artificial sense of touch seemed to play a crucial role for a full prosthesis integration and acceptance.

Summary

Artificial sensory feedback implemented in bionic limbs enhances the cognitive integration of the prosthetic device in amputees. The multisensory experience can be measured and must be considered in the design of novel somatosensory neural prostheses where the goal is to provide a realistic sensory experience to the prosthetic user. The correct integration of these sensory signals will guarantee higher-level cognitive benefits as a better prosthesis embodiment and a reduction of perceived limb distortions.