Correlation of neural activity with behavioral kinematics reveals distinct sensory encoding and evidence accumulation processes during active tactile sensing

A tensor decomposition reveals ageing-induced differences in muscle and grip-load force couplings during object lifting

Do motor patterns of object lifting movements change as a result of ageing? Here we propose a methodology for the characterization of these motor patterns across individuals of different age groups. Specifically, we employ a bimanual grasp-lift-replace protocol with younger and older adults and combine measurements of muscle activity with grip and load forces to provide a window into the motor strategies supporting effective object lifts. We introduce a tensor decomposition to identify patterns of muscle activity and grip-load force ratios while also characterizing their temporal profiles and relative activation across object weights and participants of different age groups. We then probe age-induced changes in these components. A classification analysis reveals three motor components that are differentially recruited between the two age groups. Linear regression analyses further show that advanced age and poorer manual dexterity can be predicted by the coupled activation of forearm and hand muscles which is associated with high levels of grip force. Our findings suggest that ageing may induce stronger muscle couplings in distal aspects of the upper limbs, and a less economic grasping strategy to overcome age-related decline in manual dexterity.

Modality-specific impacts of distractors on visual and auditory categorical decision-making: an evidence accumulation perspective

Our brain constantly processes multisensory inputs to make decisions and guide behaviors, but how goal-relevant processes are influenced by irrelevant information is unclear. Here, we investigated the effects of intermodal and intramodal task-irrelevant information on visual and auditory categorical decision-making. In both visual and auditory tasks, we manipulated the modality of irrelevant inputs (visual vs. auditory vs. none) and used linear discrimination analysis of EEG and hierarchical drift-diffusion modeling (HDDM) to identify when and how task-irrelevant information affected decision-relevant processing. The results revealed modality-specific impacts of irrelevant inputs on visual and auditory categorical decision-making. The distinct effects on the visual task were shown on the neural components, with auditory distractors amplifying the sensory processing whereas visual distractors amplifying the post-sensory process. Conversely, the distinct effects on the auditory task were shown in behavioral performance and underlying cognitive processes. Visual distractors facilitate behavioral performance and affect both stages, but auditory distractors interfere with behavioral performance and impact on the sensory processing rather than the post-sensory decision stage. Overall, these findings suggested that auditory distractors affect the sensory processing stage of both tasks while visual distractors affect the post-sensory decision stage of visual categorical decision-making and both stages of auditory categorical decision-making. This study provides insights into how humans process information from multiple sensory modalities during decision-making by leveraging modality-specific impacts.

Adaptive decision-making depends on pupil-linked arousal in rats performing tactile discrimination tasks

Perceptual decision-making is a dynamic cognitive process and is shaped by many factors, including behavioral state, reward contingency, and sensory environment. To understand the extent to which adaptive behavior in decision-making is dependent on pupil-linked arousal, we trained head-fixed rats to perform perceptual decision-making tasks and systematically manipulated the probability of Go and No-go stimuli while simultaneously measuring their pupil size in the tasks. Our data demonstrated that the animals adaptively modified their behavior in response to the changes in the sensory environment. The response probability to both Go and No-go stimuli decreased as the probability of the Go stimulus being presented decreased. Analyses within the signal detection theory framework showed that while the animals' perceptual sensitivity was invariant, their decision criterion increased as the probability of the Go stimulus decreased. Simulation results indicated that the adaptive increase in the decision criterion will increase possible water rewards during the task. Moreover, the adaptive decision-making is dependent on pupil-linked arousal as the increase in the decision criterion was the largest during low pupil-linked arousal periods. Taken together, our results demonstrated that the rats were able to adjust their decision-making to maximize rewards in the tasks, and that adaptive behavior in perceptual decision-making is dependent on pupil-linked arousal.NEW & NOTEWORTHY Perceptual decision-making is a dynamic cognitive process and is shaped by many factors. However, the extent to which changes in sensory environment result in adaptive decision-making remains poorly understood. Our data provided new experimental evidence demonstrating that the rats were able to adaptively modify their decision criterion to maximize water reward in response to changes in the statistics of the sensory environment. Furthermore, the adaptive decision-making is dependent on pupil-linked arousal.

Transcranial focused ultrasound stimulation of cortical and thalamic somatosensory areas in human

The effects of transcranial focused ultrasound (FUS) stimulation of the primary somatosensory cortex and its thalamic projection (i.e., ventral posterolateral nucleus) on the generation of electroencephalographic (EEG) responses were evaluated in healthy human volunteers. Stimulation of the unilateral somatosensory circuits corresponding to the non-dominant hand generated EEG evoked potentials across all participants; however, not all perceived stimulation-mediated tactile sensations of the hand. These FUS-evoked EEG potentials (FEP) were observed from both brain hemispheres and shared similarities with somatosensory evoked potentials (SSEP) from median nerve stimulation. Use of a 0.5 ms pulse duration (PD) sonication given at 70% duty cycle, compared to the use of 1 and 2 ms PD, elicited more distinctive FEP peak features from the hemisphere ipsilateral to sonication. Although several participants reported hearing tones associated with FUS stimulation, the observed FEP were not likely to be confounded by the auditory sensation based on a separate measurement of auditory evoked potentials (AEP) to tonal stimulation (mimicking the same repetition frequency as the FUS stimulation). Off-line changes in resting-state functional connectivity (FC) associated with thalamic stimulation revealed that the FUS stimulation enhanced connectivity in a network of sensorimotor and sensory integration areas, which lasted for at least more than an hour. Clinical neurological evaluations, EEG, and neuroanatomical MRI did not reveal any adverse or unintended effects of sonication, attesting its safety. These results suggest that FUS stimulation may induce long-term neuroplasticity in humans, indicating its neurotherapeutic potential for various neurological and neuropsychiatric conditions.

Neural Encoding of Active Multi-Sensing Enhances Perceptual Decision-Making via a Synergistic Cross-Modal Interaction

Most perceptual decisions rely on the active acquisition of evidence from the environment involving stimulation from multiple senses. However, our understanding of the neural mechanisms underlying this process is limited. Crucially, it remains elusive how different sensory representations interact in the formation of perceptual decisions. To answer these questions, we used an active sensing paradigm coupled with neuroimaging, multivariate analysis, and computational modeling to probe how the human brain processes multisensory information to make perceptual judgments. Participants of both sexes actively sensed to discriminate two texture stimuli using visual (V) or haptic (H) information or the two sensory cues together (VH). Crucially, information acquisition was under the participants' control, who could choose where to sample information from and for how long on each trial. To understand the neural underpinnings of this process, we first characterized where and when active sensory experience (movement patterns) is encoded in human brain activity (EEG) in the three sensory conditions. Then, to offer a neurocomputational account of active multisensory decision formation, we used these neural representations of active sensing to inform a drift diffusion model of decision-making behavior. This revealed a multisensory enhancement of the neural representation of active sensing, which led to faster and more accurate multisensory decisions. We then dissected the interactions between the V, H, and VH representations using a novel information-theoretic methodology. Ultimately, we identified a synergistic neural interaction between the two unisensory (V, H) representations over contralateral somatosensory and motor locations that predicted multisensory (VH) decision-making performance.SIGNIFICANCE STATEMENT In real-world settings, perceptual decisions are made during active behaviors, such as crossing the road on a rainy night, and include information from different senses (e.g., car lights, slippery ground). Critically, it remains largely unknown how sensory evidence is combined and translated into perceptual decisions in such active scenarios. Here we address this knowledge gap. First, we show that the simultaneous exploration of information across senses (multi-sensing) enhances the neural encoding of active sensing movements. Second, the neural representation of active sensing modulates the evidence available for decision; and importantly, multi-sensing yields faster evidence accumulation. Finally, we identify a cross-modal interaction in the human brain that correlates with multisensory performance, constituting a putative neural mechanism for forging active multisensory perception.

Neurocomputational mechanisms underlying cross-modal associations and their influence on perceptual decisions

When exposed to complementary features of information across sensory modalities, our brains formulate cross-modal associations between features of stimuli presented separately to multiple modalities. For example, auditory pitch-visual size associations map high-pitch tones with small-size visual objects, and low-pitch tones with large-size visual objects. Preferential, or congruent, cross-modal associations have been shown to affect behavioural performance, i.e. choice accuracy and reaction time (RT) across multisensory decision-making paradigms. However, the neural mechanisms underpinning such influences in perceptual decision formation remain unclear. Here, we sought to identify when perceptual improvements from associative congruency emerge in the brain during decision formation. In particular, we asked whether such improvements represent ‘early’ sensory processing benefits, or ‘late’ post-sensory changes in decision dynamics. Using a modified version of the Implicit Association Test (IAT), coupled with electroencephalography (EEG), we measured the neural activity underlying the effect of auditory stimulus-driven pitch-size associations on perceptual decision formation. Behavioural results showed that participants responded significantly faster during trials when auditory pitch was congruent, rather than incongruent, with its associative visual size counterpart. We used multivariate Linear Discriminant Analysis (LDA) to characterise the spatiotemporal dynamics of EEG activity underpinning IAT performance. We found an ‘Early’ component (∼100–110 ms post-stimulus onset) coinciding with the time of maximal discrimination of the auditory stimuli, and a ‘Late’ component (∼330–340 ms post-stimulus onset) underlying IAT performance. To characterise the functional role of these components in decision formation, we incorporated a neurally-informed Hierarchical Drift Diffusion Model (HDDM), revealing that the Late component decreases response caution, requiring less sensory evidence to be accumulated, whereas the Early component increased the duration of sensory-encoding processes for incongruent trials. Overall, our results provide a mechanistic insight into the contribution of ‘early’ sensory processing, as well as ‘late’ post-sensory neural representations of associative congruency to perceptual decision formation.

Auditory information enhances post-sensory visual evidence during rapid multisensory decision-making

Despite recent progress in understanding multisensory decision-making, a conclusive mechanistic account of how the brain translates the relevant evidence into a decision is lacking. Specifically, it remains unclear whether perceptual improvements during rapid multisensory decisions are best explained by sensory (i.e., ‘Early’) processing benefits or post-sensory (i.e., ‘Late’) changes in decision dynamics. Here, we employ a well-established visual object categorisation task in which early sensory and post-sensory decision evidence can be dissociated using multivariate pattern analysis of the electroencephalogram (EEG). We capitalize on these distinct neural components to identify when and how complementary auditory information influences the encoding of decision-relevant visual evidence in a multisensory context. We show that it is primarily the post-sensory, rather than the early sensory, EEG component amplitudes that are being amplified during rapid audiovisual decision-making. Using a neurally informed drift diffusion model we demonstrate that a multisensory behavioral improvement in accuracy arises from an enhanced quality of the relevant decision evidence, as captured by the post-sensory EEG component, consistent with the emergence of multisensory evidence in higher-order brain areas.

A conclusive account on how the brain translates audiovisual evidence into a rapid decision is still lacking. Here, using a neurally-informed modelling approach, the authors show that sounds amplify visual evidence later in the decision process, in line with higher-order multisensory effects.

Visual detection is locked to the internal dynamics of cortico-motor control

Movements overtly sample sensory information, making sensory analysis an active-sensing process. In this study, we show that visual information sampling is not just locked to the (overt) movement dynamics but to the internal (covert) dynamics of cortico-motor control. We asked human participants to perform continuous isometric contraction while detecting unrelated and unpredictable near-threshold visual stimuli. The motor output (force) shows zero-lag coherence with brain activity (recorded via electroencephalography) in the beta-band, as previously reported. In contrast, cortical rhythms in the alpha-band systematically forerun the motor output by 200 milliseconds. Importantly, visual detection is facilitated when cortico-motor alpha (not beta) synchronization is enhanced immediately before stimulus onset, namely, at the optimal phase relationship for sensorimotor communication. These findings demonstrate an ongoing coupling between visual sampling and motor control, suggesting the operation of an internal and alpha-cycling visuomotor loop.

Interplay between components of pupil-linked phasic arousal and its role in driving behavioral choice in Go/No-Go perceptual decision-making

In decision-making tasks, neural circuits involved in different aspects of information processing may activate the central arousal system, likely through their interconnection with brainstem arousal nuclei, collectively contributing to the observed pupil-linked phasic arousal. However, the individual components of the phasic arousal associated with different elements of information processing and their effects on behavior remain little known. In this study, we used machine learning techniques to decompose pupil-linked phasic arousal evoked by different components of information processing in rats performing a Go/No-Go perceptual decision-making task. We found that phasic arousal evoked by stimulus encoding was larger for the Go stimulus than the No-Go stimulus. For each session, the separation between distributions of phasic arousal evoked by the Go and by the No-Go stimulus was predictive of perceptual performance. The separation between distributions of decision-formation-evoked arousal on correct and incorrect trials was correlated with decision criterion but not perceptual performance. When a Go stimulus was presented, the action of go was primarily determined by the phasic arousal evoked by stimulus encoding. On the contrary, when a No-Go stimulus was presented, the action of go was determined by phasic arousal elicited by both stimulus encoding and decision formation. Drift diffusion modeling revealed that the four model parameters were better accounted for when phasic arousal elicited by both stimulus encoding and decision formation was considered. These results suggest that the interplay between phasic arousal evoked by both stimulus encoding and decision formation has important functional consequences on forming behavioral choice in perceptual decision-making.

Motor modules account for active perception of force

Despite longstanding evidence suggesting a relation between action and perception, the mechanisms underlying their integration are still unclear. It has been proposed that to simplify the sensorimotor integration processes underlying active perception, the central nervous system (CNS) selects patterns of movements aimed at maximizing sampling of task-related sensory input. While previous studies investigated the action-perception loop focusing on the role of higher-level features of motor behavior (e.g., kinematic invariants, effort), the present study explored and quantified the contribution of lower-level organization of motor control. We tested the hypothesis that the coordinated recruitment of group of muscles (i.e., motor modules) engaged to counteract an external force contributes to participants’ perception of the same force. We found that: 1) a model describing the modulation of a subset of motor modules involved in the motor task accounted for about 70% of participants’ perceptual variance; 2) an alternative model, incompatible with the motor modules hypothesis, accounted for significantly lower variance of participants’ detection performance. Our results provide empirical evidence of the potential role played by muscle activation patterns in active perception of force. They also suggest that a modular organization of motor control may mediate not only coordination of multiple muscles, but also perceptual inference.

Scene complexity modulates degree of feedback activity during object detection in natural scenes

Selective brain responses to objects arise within a few hundreds of milliseconds of neural processing, suggesting that visual object recognition is mediated by rapid feed-forward activations. Yet disruption of neural responses in early visual cortex beyond feed-forward processing stages affects object recognition performance. Here, we unite these discrepant findings by reporting that object recognition involves enhanced feedback activity (recurrent processing within early visual cortex) when target objects are embedded in natural scenes that are characterized by high complexity. Human participants performed an animal target detection task on natural scenes with low, medium or high complexity as determined by a computational model of low-level contrast statistics. Three converging lines of evidence indicate that feedback was selectively enhanced for high complexity scenes. First, functional magnetic resonance imaging (fMRI) activity in early visual cortex (V1) was enhanced for target objects in scenes with high, but not low or medium complexity. Second, event-related potentials (ERPs) evoked by target objects were selectively enhanced at feedback stages of visual processing (from ~220 ms onwards) for high complexity scenes only. Third, behavioral performance for high complexity scenes deteriorated when participants were pressed for time and thus less able to incorporate the feedback activity. Modeling of the reaction time distributions using drift diffusion revealed that object information accumulated more slowly for high complexity scenes, with evidence accumulation being coupled to trial-to-trial variation in the EEG feedback response. Together, these results suggest that while feed-forward activity may suffice to recognize isolated objects, the brain employs recurrent processing more adaptively in naturalistic settings, using minimal feedback for simple scenes and increasing feedback for complex scenes.

Pupil-linked arousal modulates behavior in rats performing a whisker deflection direction discrimination task

Non-luminance-mediated changes in pupil size have been widely used to index arousal state. Recent animal studies have demonstrated correlations between behavioral state-related pupil dynamics and sensory processing. However, the relationship between pupil-linked arousal and behavior in animals performing perceptual tasks has not been fully elucidated. In the present study, we trained head-fixed rats to discriminate between directions of whisker movements using a Go/No-Go discrimination paradigm while imaging their pupils. Reaction times in this discrimination task were significantly slower than in previously reported detection tasks with similar setup, suggesting that discrimination required an increased cognitive load. We found the pupils dilated for all trials following stimulus presentation. Interestingly, in correct rejection trials, where pupil dilations solely resulted from cognitive processing, dilations were larger for more difficult stimuli. Baseline pupil size before stimulus presentation strongly correlated with behavior, as perceptual sensitivity peaked at intermediate pupil baselines and reaction time was fastest at large baselines. We further explored these relationships by investigating to what extent pupil baseline was predictive of upcoming behavior and found that a Bayesian decoder had significantly greater-than-chance probability in correctly predicting behavioral outcomes. Moreover, the outcome of the previous trial showed a strong correlation with behavior on present trials. Animals were more liberal and faster in responding following hit trials, whereas perceptual sensitivity was greatest following correct rejection trials. Taken together, these results suggest a tight correlation between pupil dynamics, perceptual performance, and reaction time in behaving rats, all of which are modulated by fluctuating arousal state. NEW & NOTEWORTHY In this study, we for the first time demonstrated that head-fixed rats were able to discriminate different directions of whisker movement. Interestingly, we found that the pupil dilated more when discriminating more difficult stimuli, a phenomenon reported in human subjects but not in animals. Baseline pupil size before stimulus presentation was found to strongly correlate with behavior, and a Bayesian decoder had significantly greater-than-chance probability in correctly predicting behavioral outcomes based on the baseline pupil size.