Visual Feedback Dominates the Sense of Agency for Brain-Machine Actions

Influence of feedback transparency on motor imagery neurofeedback performance: the contribution of agency

Objective.Neurofeedback (NF) is a cognitive training procedure based on real-time feedback (FB) of a participant's brain activity that they must learn to self-regulate. A classical visual FB delivered in a NF task is a filling gauge reflecting a measure of brain activity. This abstract visual FB is not transparently linked-from the subject's perspective-to the task performed (e.g., motor imagery (MI)). This may decrease the sense of agency, that is, the participants' reported control over FB. Here, we assessed the influence of FB transparency on NF performance and the role of agency in this relationship.Approach.Participants performed a NF task using MI to regulate brain activity measured using electroencephalography. In separate blocks, participants experienced three different conditions designed to vary transparency: FB was presented as either (1) a swinging pendulum, (2) a clenching virtual hand, (3) a clenching virtual hand combined with a motor illusion induced by tendon vibration. We measured self-reported agency and user experience after each NF block.Main results. We found that FB transparency influences NF performance. Transparent visual FB provided by the virtual hand resulted in significantly better NF performance than the abstract FB of the pendulum. Surprisingly, adding a motor illusion to the virtual hand significantly decreased performance relative to the virtual hand alone. When introduced in incremental linear mixed effect models, self-reported agency was significantly associated with NF performance and it captured the variance related to the effect of FB transparency on NF performance.Significance. Our results highlight the relevance of transparent FB in relation to the sense of agency. This is likely an important consideration in designing FB to improve NF performance and learning outcomes.

Real-Time Detection and Feedback of Canonical Electroencephalogram Microstates: Validating a Neurofeedback System as a Function of Delay

Recent neurotechnology has developed various methods for neurofeedback (NF), in which participants observe their own neural activity to be regulated in an ideal direction. EEG-microstates (EEGms) are spatially featured states that can be regulated through NF training, given that they have recently been indicated as biomarkers for some disorders. The current study was conducted to develop an EEG-NF system for detecting “canonical 4 EEGms” in real time. There are four representative EEG states, regardless of the number of channels, preprocessing procedures, or participants. Accordingly, our 10 Hz NF system was implemented to detect them (msA, B, C, and D) and audio-visually inform participants of its detection. To validate the real-time effect of this system on participants’ performance, the NF was intentionally delayed for participants to prevent their cognitive control in learning. Our results suggest that the feedback effect was observed only under the no-delay condition. The number of Hits increased significantly from the baseline period and increased from the 1- or 20-s delay conditions. In addition, when the Hits were compared among the msABCD, each cognitive or perceptual function could be characterized, though the correspondence between each microstate and psychological ability might not be that simple. For example, msD should be generally task-positive and less affected by the inserted delay, whereas msC is more delay-sensitive. In this study, we developed and validated a new EEGms-NF system as a function of delay. Although the participants were naive to the inserted delay, the real-time NF successfully increased their Hit performance, even within a single-day experiment, although target specificity remains unclear. Future research should examine long-term training effects using this NF system.

Sense of agency for intracortical brain-machine interfaces

Intracortical brain-machine interfaces decode motor commands from neural signals and translate them into actions, enabling movement for paralysed individuals. The subjective sense of agency associated with actions generated via intracortical brain-machine interfaces, the neural mechanisms involved and its clinical relevance are currently unknown. By experimentally manipulating the coherence between decoded motor commands and sensory feedback in a tetraplegic individual using a brain-machine interface, we provide evidence that primary motor cortex processes sensory feedback, sensorimotor conflicts and subjective states of actions generated via the brain-machine interface. Neural signals processing the sense of agency affected the proficiency of the brain-machine interface, underlining the clinical potential of the present approach. These findings show that primary motor cortex encodes information related to action and sensing, but also sensorimotor and subjective agency signals, which in turn are relevant for clinical applications of brain-machine interfaces.

"Mine Works Better": Examining the Influence of Embodiment in Virtual Reality on the Sense of Agency During a Binary Motor Imagery Task With a Brain-Computer Interface

Motor imagery-based brain-computer interfaces (MI-BCI) have been proposed as a means for stroke rehabilitation, which combined with virtual reality allows for introducing game-based interactions into rehabilitation. However, the control of the MI-BCI may be difficult to obtain and users may face poor performance which frustrates them and potentially affects their motivation to use the technology. Decreases in motivation could be reduced by increasing the users' sense of agency over the system. The aim of this study was to understand whether embodiment (ownership) of a hand depicted in virtual reality can enhance the sense of agency to reduce frustration in an MI-BCI task. Twenty-two healthy participants participated in a within-subject study where their sense of agency was compared in two different embodiment experiences: 1) avatar hand (with body), or 2) abstract blocks. Both representations closed with a similar motion for spatial congruency and popped a balloon as a result. The hand/blocks were controlled through an online MI-BCI. Each condition consisted of 30 trials of MI-activation of the avatar hand/blocks. After each condition a questionnaire probed the participants' sense of agency, ownership, and frustration. Afterwards, a semi-structured interview was performed where the participants elaborated on their ratings. Both conditions supported similar levels of MI-BCI performance. A significant correlation between ownership and agency was observed (r = 0.47, p = 0.001). As intended, the avatar hand yielded much higher ownership than the blocks. When controlling for performance, ownership increased sense of agency. In conclusion, designers of BCI-based rehabilitation applications can draw on anthropomorphic avatars for the visual mapping of the trained limb to improve ownership. While not While not reducing frustration ownership can improve perceived agency given sufficient BCI performance. In future studies the findings should be validated in stroke patients since they may perceive agency and ownership differently than able-bodied users.

Sense of agency for mental actions: Insights from a belief-based action-effect paradigm

A substantial body of research has converged on the idea that the sense of agency arises from the integration of multiple sources of information. In this study, we investigated whether a measurable sense of agency can be detected for mental actions, without the contribution of motor components. We used a fake action-effect paradigm, where participants were led to think that a motor action or a particular thought could trigger a sound. Results showed that the sense of agency, when measured through explicit reports, was of comparable strength for motor and mental actions. The intentional binding effect, a phenomenon typically associated with the experience of agency, was also observed for both motor and mental actions. Taken together, our results provide novel insights into the specific role of intentional cues in instantiating a sense of agency, even in the absence of motor signals.

A multi-modal modified feedback self-paced BCI to control the gait of an avatar

Brain-computer interfaces (BCI) have been used to control the gait of a virtual self-avatar with a proposed application in the field of gait rehabilitation.

OBJECTIVE

to develop a high performance multi-modal BCI to control single steps and forward walking of an immersive virtual reality avatar. This system will overcome the limitation of existing systems.

APPROACH

This system used MI of these actions, in cue-paced and self-paced modes. Twenty healthy participants participated in this 4 sessions study across 4 different days. They were cued to imagine a single step forward with their right or left foot, or to imagine walking forward. They were instructed to reach a target by using the MI of multiple steps (self-paced switch-control mode) or by maintaining MI of forward walking (continuous-control mode). The movement of the avatar was controlled by two calibrated RLDA classifiers that used the µ power spectral density (PSD) over the foot area of the motor cortex as a feature. The classifiers were retrained after every session. For a subset of the trials, positive modified feedback was presented to half of the participants.

MAIN RESULTS

All participants were able to operate the BCI. Their average offline performance, after retraining the classifiers was 86.0 ± 6.1%, showing that the recalibration of the classifiers enhanced the offline performance of the BCI (p < 0.01). The average online performance was 85.9 ± 8.4% showing that modified feedback enhanced BCI performance (p =0.001). The average performance was 83% at self-paced switch control and 92% at continuous control mode.

SIGNIFICANCE

This study reports on the first novel integration of different design approaches, different control modes and different performance enhancement techniques, all in parallel in one single high performance and multi-modal BCI system, to control single steps and forward walking of an immersive virtual reality avatar.

Control Modification of Grasp Force Covaries Agency and Performance on Rigid and Compliant Surfaces

This study investigated how modifications in the display of a computer trace under user control of grasp forces can co-modulate agency (perception of control) and performance of grasp on rigid and compliant surfaces. We observed positive correlation (p < 0.01) between implicit agency, measured from time-interval estimation for intentional binding, and grasp performance, measured by force-tracking error, across varying control modes for each surface type. The implications of this work are design directives for cognition-centered device interfaces for rehabilitation of grasp after neurotraumas such as spinal cord and brain injuries while considering if grasp interaction is rigid or compliant. These device interfaces should increase user integration to virtual reality training and powered assistive devices such as exoskeletons and prostheses. The modifications in control modes for this study included changes in force magnitude, addition of mild noise, and a measure of automation. Significant differences (p < 0.001) were observed for each surface type across control modes with metrics for implicit agency, performance, and grasp control efficiency. Explicit agency, measured from user survey responses, did not exhibit significant variations in this study, suggesting implicit measures of agency are needed for identifying co-modulation with grasp performance. Grasp on the compliant surface resulted in greater dependence of performance on agency and increases in agency and performance with the addition of mild noise. Noise in conjunction with perceived freedom at a flexible surface may have amplified visual feedback responses. Introducing automation in control decreased agency and performance for both surfaces, suggesting the value in continuous user control of grasp. In conclusion, agency and performance of grasp can be co-modulated across varying modes of control, especially for compliant grasp actions. Future studies should consider reliable measures of implicit agency, including physiological recordings, to automatically adapt rehabilitation interfaces for better cognitive engagement and to accelerate functional outcomes.

Modified sensory feedback enhances the sense of agency during continuous body movements in virtual reality

The sense of agency refers to the feeling of control over one's own actions, and through them, the external events. This study examined the effect of modified visual feedback on the sense of agency over one's body movements using virtual reality in healthy individuals whose motor control was disturbed. Participants moved a virtual object using their right hand to trace a trajectory (Experiment 1) or a leading target (Experiment 2). Their motor control was disturbed by a delay in visual feedback (Experiment 1) or a 1-kg weight attached to their wrist (Experiment 2). In the offset conditions, the virtual object was presented at the median point between the desired position and the participants' actual hand position. In both experiments, participants reported improved sense of agency in the offset condition compared to the aligned condition where the visual feedback reflected their actual body movements, despite their motion being less precise in the offset condition. The results show that sense of agency can be enhanced by modifying feedback to motor tasks according to the goal of the task, even when visual feedback is discrepant from the actual body movements. The present study sheds light on the possibility of artificially enhancing body agency to improve voluntary motor control.

How using brain-machine interfaces influences the human sense of agency

Brain-machine interfaces (BMI) allows individuals to control an external device by controlling their own brain activity, without requiring bodily or muscle movements. Performing voluntary movements is associated with the experience of agency ("sense of agency") over those movements and their outcomes. When people voluntarily control a BMI, they should likewise experience a sense of agency. However, using a BMI to act presents several differences compared to normal movements. In particular, BMIs lack sensorimotor feedback, afford lower controllability and are associated with increased cognitive fatigue. Here, we explored how these different factors influence the sense of agency across two studies in which participants learned to control a robotic hand through motor imagery decoded online through electroencephalography. We observed that the lack of sensorimotor information when using a BMI did not appear to influence the sense of agency. We further observed that experiencing lower control over the BMI reduced the sense of agency. Finally, we observed that the better participants controlled the BMI, the greater was the appropriation of the robotic hand, as measured by body-ownership and agency scores. Results are discussed based on existing theories on the sense of agency in light of the importance of BMI technology for patients using prosthetic limbs.

Short-delay neurofeedback facilitates training of the parietal alpha rhythm

OBJECTIVE

Feedback latency was shown to be a critical parameter in a range of applications that imply learning. The therapeutic effects of neurofeedback (NFB) remain controversial. We hypothesized that often encountered unreliable results of NFB intervention could be associated with large feedback latency values that are often uncontrolled and may preclude the efficient learning.

APPROACH

We engaged our subjects into a parietal alpha power unpregulating paradigm faciliated by visual neurofeedback based on the invidually extracted envelope of the alpha-rhythm at P4 electrode. NFB was displayed either as soon as EEG envelope was processed, or with an extra 250 or 500-ms delay. The feedback training consisted of 15 two-minute long blocks interleaved with 15s pauses. We have also recorded two minute long baselines immediately before and after the training.

MAIN RESULTS

The time course of NFB-induced changes in the alpha rhythm power clearly depended on NFB latency, as shown with the adaptive Neyman test. NFB had a strong effect on the alpha-spindle incidence rate, but not on their duration or amplitude. The sustained changes in alpha activity measured after the completion of NFB training were negatively correlated to latency, with the maximum change for the shortest tested latency and no change for the longest.

SIGNIFICANCE

Here we for the first time show that visual NFB of parietal electroencephalographic (EEG) alpha-activity is efficient only when delivered to human subjects at short latency, which guarantees that NFB arrives when an alpha spindle is still ongoing. Such a considerable effect of NFB latency on the alpha-activity temporal structure could explain some of the previous inconsistent results, where latency was neither controlled nor documented. Clinical practitioners and manufacturers of NFB equipment should add latency to their specifications while enabling latency monitoring and supporting short-latency operations.

Digital filters for low-latency quantification of brain rhythms in real time

OBJECTIVE

The rapidly developing paradigm of closed-loop neuroscience has extensively employed brain rhythms as the signal forming real-time neurofeedback, triggering brain stimulation, or governing stimulus selection. However, the efficacy of brain rhythm contingent paradigms suffers from significant delays related to the process of extraction of oscillatory parameters from broad-band neural signals with conventional methods. To this end, real-time algorithms are needed that would shorten the delay while maintaining an acceptable speed-accuracy trade-off.

APPROACH

Here we evaluated a family of techniques based on the application of the least-squares complex-valued filter (LSCF) design to real-time quantification of brain rhythms. These techniques allow for explicit optimization of the speed-accuracy trade-off when quantifying oscillatory patterns. We used EEG data collected from 10 human participants to systematically compare LSCF approach to the other commonly used algorithms. Each method being evaluated was optimized by scanning through the grid of its hyperparameters using independent data samples.

MAIN RESULTS

When applied to the task of estimating oscillatory envelope and phase, the LSCF techniques outperformed in speed and accuracy both conventional Fourier transform and rectification based methods as well as more advanced techniques such as those that exploit autoregressive extrapolation of narrow-band filtered signals. When operating at zero latency, the weighted LSCF approach yielded 75% accuracy when detecting alpha-activity episodes, as defined by the amplitude crossing of the 95th-percentile threshold.

SIGNIFICANCE

The LSCF approaches are easily applicable to low-delay quantification of brain rhythms. As such, these methods are useful in a variety of neurofeedback, brain-computer-interface and other experimental paradigms that require rapid monitoring of brain rhythms.

A Usability Study of Low-cost Wireless Brain-Computer Interface for Cursor Control Using Online Linear Model

Computer cursor control using electroencephalogram (EEG) signals is a common and well-studied brain-computer interface (BCI). The emphasis of the literature has been primarily on evaluation of the objective measures of assistive BCIs such as accuracy of the neural decoder whereas the subjective measures such as user's satisfaction play an essential role for the overall success of a BCI. As far as we know, the BCI literature lacks a comprehensive evaluation of the usability of the mind-controlled computer cursor in terms of decoder efficiency (accuracy), user experience, and relevant confounding variables concerning the platform for the public use. To fill this gap, we conducted a two-dimensional EEG-based cursor control experiment among 28 healthy participants. The computer cursor velocity was controlled by the imagery of hand movement using a paradigm presented in the literature named imagined body kinematics (IBK) with a low-cost wireless EEG headset. We evaluated the usability of the platform for different objective and subjective measures while we investigated the extent to which the training phase may influence the ultimate BCI outcome. We conducted pre- and post- BCI experiment interview questionnaires to evaluate the usability. Analyzing the questionnaires and the testing phase outcome shows a positive correlation between the individuals' ability of visualization and their level of mental controllability of the cursor. Despite individual differences, analyzing training data shows the significance of electrooculogram (EOG) on the predictability of the linear model. The results of this work may provide useful insights towards designing a personalized user-centered assistive BCI.

Agency and Performance of Reach-to-Grasp With Modified Control of a Virtual Hand: Implications for Rehabilitation

This study investigated how modified control of a virtual hand executing reach-to-grasp affects functional performance and agency (perception of control). The objective of this work was to demonstrate positive relationships between reaching performance and grasping agency and motivate greater consideration of agency in movement rehabilitation. We hypothesized that agency and performance have positive correlation across varying control modes of the virtual hand. In this study, each participant controlled motion of a virtual hand through motion of his or her own hand. Control of the virtual hand was modified according to a specific control mode. Each mode involved the virtual hand moving at a modified speed, having noise, or including a level of automation. These specific modes represent potential control features to adapt for a rehabilitation device such as a prosthetic arm and hand. In this study, significant changes in agency and performance were observed across the control modes. Overall, a significant positive relationship (p < 0.001) was observed between the primary performance metric of reach (tracking a minimum path length trajectory) and an implicit measurement of agency (intentional binding). Intentional binding was assessed through participant perceptions of time-intervals between grasp contact and a sound event. Other notable findings include improved movement efficiency (increased smoothness, reduced acceleration) during expression of higher agency and shift toward greater implicit versus explicit agency with higher control speed. Positively relating performance and agency incentivizes control adaptation of powered movement devices, such as prostheses or exoskeletons, to maximize both user engagement and functional performance. Agency-based approaches may foster user-device integration at a cognitive level and facilitate greater clinical retention of the device. Future work should identify robust and automated methods to adapt device control for increased agency. Objectives include how virtual reality (VR) may identify optimal control of real-world devices and assessing real-time agency from neurophysiological signals.

Brain-computer interfaces and virtual reality for neurorehabilitation

Brain-computer interfaces (BCIs) and virtual reality (VR) are two technologic advances that are changing our way of interacting with the world. BCIs can be used to influence and can serve as a control mechanism in navigation tasks, communication, or other assistive functions. VR can create ad hoc interactive scenarios that involve all our senses, stimulate the brain in a multisensory fashion, and increase the motivation and fun with game-like environments. VR and motion tracking enable natural human-computer interaction at cognitive and physical levels. This includes both brain and body in the design of meaningful VR experiences; these cases in which participants feel naturally present could help augment the benefits of BCIs for assistive and neurorehabilitation applications for the relearning of motor and cognitive skills. VR technology is now available at the consumer level thanks to the proliferation of affordable head-mounted displays (HMDs). Merging both technologies into simplified, practical devices may help democratize these technologies.

Agency and responsibility over virtual movements controlled through different paradigms of brain-computer interface

KEY POINTS

Embodiment of a virtual body was induced and its movements were controlled by two different brain-computer interface (BCI) paradigms - one based on signals from sensorimotor versus one from visual cortical areas. BCI-control of movements engenders agency, but not equally for all paradigms. Cortical sensorimotor activation correlates with agency and responsibility. This has significant implications for neurological rehabilitation and neuroethics.

ABSTRACT

Agency is the attribution of an action to the self and is a prerequisite for experiencing responsibility over its consequences. Here we investigated agency and responsibility by studying the control of movements of an embodied avatar, via brain-computer interface (BCI) technology, in immersive virtual reality. After induction of virtual body ownership by visuomotor correlations, healthy participants performed a motor task with their virtual body. We compared the passive observation of the subject's 'own' virtual arm performing the task with (1) the control of the movement through activation of sensorimotor areas (motor imagery) and (2) the control of the movement through activation of visual areas (steady-state visually evoked potentials). The latter two conditions were carried out using a BCI and both shared the intention and the resulting action. We found that BCI-control of movements engenders the sense of agency, which is strongest for sensorimotor area activation. Furthermore, increased activity of sensorimotor areas, as measured using EEG, correlates with levels of agency and responsibility. We discuss the implications of these results for the neural basis of agency.

Progressive Training for Motor Imagery Brain-Computer Interfaces Using Gamification and Virtual Reality Embodiment

This paper presents a gamified motor imagery brain-computer interface (MI-BCI) training in immersive virtual reality. The aim of the proposed training method is to increase engagement, attention, and motivation in co-adaptive event-driven MI-BCI training. This was achieved using gamification, progressive increase of the training pace, and virtual reality design reinforcing body ownership transfer (embodiment) into the avatar. From the 20 healthy participants performing 6 runs of 2-class MI-BCI training (left/right hand), 19 were trained for a basic level of MI-BCI operation, with average peak accuracy in the session = 75.84%. This confirms the proposed training method succeeded in improvement of the MI-BCI skills; moreover, participants were leaving the session in high positive affect. Although the performance was not directly correlated to the degree of embodiment, subjective magnitude of the body ownership transfer illusion correlated with the ability to modulate the sensorimotor rhythm.

A Characterization of Brain-Computer Interface Performance Trade-Offs Using Support Vector Machines and Deep Neural Networks to Decode Movement Intent

Laboratory demonstrations of brain-computer interface (BCI) systems show promise for reducing disability associated with paralysis by directly linking neural activity to the control of assistive devices. Surveys of potential users have revealed several key BCI performance criteria for clinical translation of such a system. Of these criteria, high accuracy, short response latencies, and multi-functionality are three key characteristics directly impacted by the neural decoding component of the BCI system, the algorithm that translates neural activity into control signals. Building a decoder that simultaneously addresses these three criteria is complicated because optimizing for one criterion may lead to undesirable changes in the other criteria. Unfortunately, there has been little work to date to quantify how decoder design simultaneously affects these performance characteristics. Here, we systematically explore the trade-off between accuracy, response latency, and multi-functionality for discrete movement classification using two different decoding strategies-a support vector machine (SVM) classifier which represents the current state-of-the-art for discrete movement classification in laboratory demonstrations and a proposed deep neural network (DNN) framework. We utilized historical intracortical recordings from a human tetraplegic study participant, who imagined performing several different hand and finger movements. For both decoders, we found that response time increases (i.e., slower reaction) and accuracy decreases as the number of functions increases. However, we also found that both the increase of response times and the decline in accuracy with additional functions is less for the DNN than the SVM. We also show that data preprocessing steps can affect the performance characteristics of the two decoders in drastically different ways. Finally, we evaluated the performance of our tetraplegic participant using the DNN decoder in real-time to control functional electrical stimulation (FES) of his paralyzed forearm. We compared his performance to that of able-bodied participants performing the same task, establishing a quantitative target for ideal BCI-FES performance on this task. Cumulatively, these results help quantify BCI decoder performance characteristics relevant to potential users and the complex interactions between them.

Meeting brain-computer interface user performance expectations using a deep neural network decoding framework

Brain-computer interface (BCI) neurotechnology has the potential to reduce disability associated with paralysis by translating neural activity into control of assistive devices^1-9. Surveys of potential end-users have identified key BCI system features^10-14, including high accuracy, minimal daily setup, rapid response times, and multifunctionality. These performance characteristics are primarily influenced by the BCI's neural decoding algorithm^1,15, which is trained to associate neural activation patterns with intended user actions. Here, we introduce a new deep neural network¹⁶ decoding framework for BCI systems enabling discrete movements that addresses these four key performance characteristics. Using intracortical data from a participant with tetraplegia, we provide offline results demonstrating that our decoder is highly accurate, sustains this performance beyond a year without explicit daily retraining by combining it with an unsupervised updating procedure^3,17-20, responds faster than competing methods⁸, and can increase functionality with minimal retraining by using a technique known as transfer learning²¹. We then show that our participant can use the decoder in real-time to reanimate his paralyzed forearm with functional electrical stimulation (FES), enabling accurate manipulation of three objects from the grasp and release test (GRT)²². These results demonstrate that deep neural network decoders can advance the clinical translation of BCI technology.

Voluntarily controlled but not merely observed visual feedback affects postural sway

Online stabilization of human standing posture utilizes multisensory afferences (e.g., vision). Whereas visual feedback of spontaneous postural sway can stabilize postural control especially when observers concentrate on their body and intend to minimize postural sway, the effect of intentional control of visual feedback on postural sway itself remains unclear. This study assessed quiet standing posture in healthy adults voluntarily controlling or merely observing visual feedback. The visual feedback (moving square) had either low or high gain and was either horizontally flipped or not. Participants in the voluntary-control group were instructed to minimize their postural sway while voluntarily controlling visual feedback, whereas those in the observation group were instructed to minimize their postural sway while merely observing visual feedback. As a result, magnified and flipped visual feedback increased postural sway only in the voluntary-control group. Furthermore, regardless of the instructions and feedback manipulations, the experienced sense of control over visual feedback positively correlated with the magnitude of postural sway. We suggest that voluntarily controlled, but not merely observed, visual feedback is incorporated into the feedback control system for posture and begins to affect postural sway.

Revisiting the link between body and agency: visual movement congruency enhances intentional binding but is not body-specific

Embodiment and agency are key aspects of how we perceive ourselves that have typically been associated with independent mechanisms. Recent work, however, has suggested that these mechanisms are related. The sense of agency arises from recognising a causal influence on the external world. This influence is typically realised through bodily movements and thus the perception of the bodily self could also be crucial for agency. We investigated whether a key index of agency - intentional binding - was modulated by body-specific information. Participants judged the interval between pressing a button and a subsequent tone. We used virtual reality to manipulate two aspects of movement feedback. First, form: participants viewed a virtual hand or sphere. Second, movement congruency: the viewed object moved congruently or incongruently with the participant's hidden hand. Both factors, form and movement congruency, significantly influenced embodiment. However, only movement congruency influenced intentional binding. Binding was increased for congruent compared to incongruent movement feedback irrespective of form. This shows that the comparison between viewed and performed movements provides an important cue for agency, whereas body-specific visual form does not. We suggest that embodiment and agency mechanisms both depend on comparisons across sensorimotor signals but that they are influenced by distinct factors.

Self-regulation strategy, feedback timing and hemodynamic properties modulate learning in a simulated fMRI neurofeedback environment

Direct manipulation of brain activity can be used to investigate causal brain-behavior relationships. Current noninvasive neural stimulation techniques are too coarse to manipulate behaviors that correlate with fine-grained spatial patterns recorded by fMRI. However, these activity patterns can be manipulated by having people learn to self-regulate their own recorded neural activity. This technique, known as fMRI neurofeedback, faces challenges as many participants are unable to self-regulate. The causes of this non-responder effect are not well understood due to the cost and complexity of such investigation in the MRI scanner. Here, we investigated the temporal dynamics of the hemodynamic response measured by fMRI as a potential cause of the non-responder effect. Learning to self-regulate the hemodynamic response involves a difficult temporal credit-assignment problem because this signal is both delayed and blurred over time. Two factors critical to this problem are the prescribed self-regulation strategy (cognitive or automatic) and feedback timing (continuous or intermittent). Here, we sought to evaluate how these factors interact with the temporal dynamics of fMRI without using the MRI scanner. We first examined the role of cognitive strategies by having participants learn to regulate a simulated neurofeedback signal using a unidimensional strategy: pressing one of two buttons to rotate a visual grating that stimulates a model of visual cortex. Under these conditions, continuous feedback led to faster regulation compared to intermittent feedback. Yet, since many neurofeedback studies prescribe implicit self-regulation strategies, we created a computational model of automatic reward-based learning to examine whether this result held true for automatic processing. When feedback was delayed and blurred based on the hemodynamics of fMRI, this model learned more reliably from intermittent feedback compared to continuous feedback. These results suggest that different self-regulation mechanisms prefer different feedback timings, and that these factors can be effectively explored and optimized via simulation prior to deployment in the MRI scanner.

By repeatedly stimulating fine-grained patterns of neural activity, it is possible to manipulate behaviors associated with these patterns. While millimeter-scale patterns cannot yet be targeted with noninvasive brain stimulation, some people can learn to self-stimulate these activity patterns if they receive real-time feedback of their own recorded brain activity through a procedure known as fMRI neurofeedback. Other ‘non-responders’ are, for reasons unknown, unable to learn how to self-regulate these patterns. Here, we investigate how the properties of the fMRI signal, feedback timing, and self-regulation strategies may lead to this non-responder effect. The signal recorded by fMRI is related to blood flow in the brain and can be delayed by up to six seconds relative to underlying neural activity, which makes it difficult to learn. Because experiments in the MRI scanner are costly and time-consuming, we created a simulated neurofeedback environment to compare continuous versus intermittent feedback timing and cognitive versus automatic self-regulation strategies. In a cognitive experiment with human participants playing a simple game with the simulated neurofeedback signal, we found continuous feedback led to faster learning. However, in a computer model of automatic reward-based learning, we found that intermittent feedback was more reliable. These results will help improve future fMRI neurofeedback experiments and treatments by improving the efficacy of neurofeedback training procedures.

Integrating prospective and retrospective cues to the sense of agency: a multi-study investigation

Sense of agency (SoA) refers to the experience of voluntary control over one’s own actions, and, through them, over events in the outside world. Recent accounts suggest that SoA involves an integration of various cues. These include prospective cues, for example, related to the fluency of action selection, and retrospective cues, linked to outcome monitoring. It remains unclear whether these cues may have independent effects on SoA, and, in particular, how their relative contributions may change during instrumental learning. In the present study, we explored these issues by conducting a multi-study analysis of seven published and unpublished studies on the role of prospective cues to the SoA. Our main question was how the effects of selection fluency on SoA might change as information about action–outcome contingencies is gathered. Results show that selection fluency can have a general and consistent influence on the SoA, independent of outcome monitoring. This suggests selection fluency is used as a heuristic cue, to prospectively inform our SoA. In addition, our results show that the influence of selection fluency on SoA may change systematically as action–outcome contingencies are gradually learned. We speculate that dysfluent selection may impair formation of mental associations between action and outcome.

Cortical and subcortical mechanisms of brain-machine interfaces

Technical advances in the field of Brain-Machine Interfaces (BMIs) enable users to control a variety of external devices such as robotic arms, wheelchairs, virtual entities and communication systems through the decoding of brain signals in real time. Most BMI systems sample activity from restricted brain regions, typically the motor and premotor cortex, with limited spatial resolution. Despite the growing number of applications, the cortical and subcortical systems involved in BMI control are currently unknown at the whole-brain level. Here, we provide a comprehensive and detailed report of the areas active during on-line BMI control. We recorded functional magnetic resonance imaging (fMRI) data while participants controlled an EEG-based BMI inside the scanner. We identified the regions activated during BMI control and how they overlap with those involved in motor imagery (without any BMI control). In addition, we investigated which regions reflect the subjective sense of controlling a BMI, the sense of agency for BMI-actions. Our data revealed an extended cortical-subcortical network involved in operating a motor-imagery BMI. This includes not only sensorimotor regions but also the posterior parietal cortex, the insula and the lateral occipital cortex. Interestingly, the basal ganglia and the anterior cingulate cortex were involved in the subjective sense of controlling the BMI. These results inform basic neuroscience by showing that the mechanisms of BMI control extend beyond sensorimotor cortices. This knowledge may be useful for the development of BMIs that offer a more natural and embodied feeling of control for the user. Hum Brain Mapp 38:2971-2989, 2017. © 2017 Wiley Periodicals, Inc.

A New Neurocognitive Interpretation of Shoulder Position Sense during Reaching: Unexpected Competence in the Measurement of Extracorporeal Space

Background. The position sense of the shoulder joint is important during reaching. Objective. To examine the existence of additional competence of the shoulder with regard to the ability to measure extracorporeal space, through a novel approach, using the shoulder proprioceptive rehabilitation tool (SPRT), during reaching. Design. Observational case-control study. Methods. We examined 50 subjects: 25 healthy and 25 with impingement syndrome with a mean age [years] of 64.52 +/− 6.98 and 68.36 +/− 6.54, respectively. Two parameters were evaluated using the SPRT: the integration of visual information and the proprioceptive afferents of the shoulder (Test 1) and the discriminative proprioceptive capacity of the shoulder, with the subject blindfolded (Test 2). These tasks assessed the spatial error (in centimeters) by the shoulder joint in reaching movements on the sagittal plane. Results. The shoulder had proprioceptive features that allowed it to memorize a reaching position and reproduce it (error of 1.22 cm to 1.55 cm in healthy subjects). This ability was lower in the impingement group, with a statistically significant difference compared to the healthy group (p < 0.05 by Mann–Whitney test). Conclusions. The shoulder has specific expertise in the measurement of the extracorporeal space during reaching movements that gradually decreases in impingement syndrome.

What Is the Sense of Agency and Why Does it Matter?

Sense of agency refers to the feeling of control over actions and their consequences. In this article I summarize what we currently know about sense of agency; looking at how it is measured and what theories there are to explain it. I then explore some of the potential applications of this research, something that the sense of agency research field has been slow to identify and implement. This is a pressing concern given the increasing importance of ‘research impact.’

Ownership and Agency of an Independent Supernumerary Hand Induced by an Imitation Brain-Computer Interface

To study body ownership and control, illusions that elicit these feelings in non-body objects are widely used. Classically introduced with the Rubber Hand Illusion, these illusions have been replicated more recently in virtual reality and by using brain-computer interfaces. Traditionally these illusions investigate the replacement of a body part by an artificial counterpart, however as brain-computer interface research develops it offers us the possibility to explore the case where non-body objects are controlled in addition to movements of our own limbs. Therefore we propose a new illusion designed to test the feeling of ownership and control of an independent supernumerary hand. Subjects are under the impression they control a virtual reality hand via a brain-computer interface, but in reality there is no causal connection between brain activity and virtual hand movement but correct movements are observed with 80% probability. These imitation brain-computer interface trials are interspersed with movements in both the subjects' real hands, which are in view throughout the experiment. We show that subjects develop strong feelings of ownership and control over the third hand, despite only receiving visual feedback with no causal link to the actual brain signals. Our illusion is crucially different from previously reported studies as we demonstrate independent ownership and control of the third hand without loss of ownership in the real hands.