Dual pathways for haptic and visual perception of spatial and texture information

Distinct but related abilities for visual and haptic object recognition

People vary in their ability to recognize objects visually. Individual differences for matching and recognizing objects visually is supported by a domain-general ability capturing common variance across different tasks (e.g., Richler et al., Psychological Review, 126, 226-251, 2019). Behavioral (e.g., Cooke et al., Neuropsychologia, 45, 484-495, 2007) and neural evidence (e.g., Amedi, Cerebral Cortex, 12, 1202-1212, 2002) suggest overlapping mechanisms in the processing of visual and haptic information in the service of object recognition, but it is unclear whether such group-average results generalize to individual differences. Psychometrically validated measures are required, which have been lacking in the haptic modality. We investigate whether object recognition ability is specific to vision or extends to haptics using psychometric measures we have developed. We use multiple visual and haptic tests with different objects and different formats to measure domain-general visual and haptic abilities and to test for relations across them. We measured object recognition abilities using two visual tests and four haptic tests (two each for two kinds of haptic exploration) in 97 participants. Partial correlation and confirmatory factor analyses converge to support the existence of a domain-general haptic object recognition ability that is moderately correlated with domain-general visual object recognition ability. Visual and haptic abilities share about 25% of their variance, supporting the existence of a multisensory domain-general ability while leaving a substantial amount of residual variance for modality-specific abilities. These results extend our understanding of the structure of object recognition abilities; while there are mechanisms that may generalize across categories, tasks, and modalities, there are still other mechanisms that are distinct between modalities.

Learning and navigating digitally rendered haptic spatial layouts

Learning spatial layouts and navigating through them rely not simply on sight but rather on multisensory processes, including touch. Digital haptics based on ultrasounds are effective for creating and manipulating mental images of individual objects in sighted and visually impaired participants. Here, we tested if this extends to scenes and navigation within them. Using only tactile stimuli conveyed via ultrasonic feedback on a digital touchscreen (i.e., a digital interactive map), 25 sighted, blindfolded participants first learned the basic layout of an apartment based on digital haptics only and then one of two trajectories through it. While still blindfolded, participants successfully reconstructed the haptically learned 2D spaces and navigated these spaces. Digital haptics were thus an effective means to learn and translate, on the one hand, 2D images into 3D reconstructions of layouts and, on the other hand, navigate actions within real spaces. Digital haptics based on ultrasounds represent an alternative learning tool for complex scenes as well as for successful navigation in previously unfamiliar layouts, which can likely be further applied in the rehabilitation of spatial functions and mitigation of visual impairments.

Decoding Reach Direction in Early "Visual" Cortex of Congenitally Blind Individuals

Motor actions, such as reaching or grasping, can be decoded from fMRI activity of early visual cortex (EVC) in sighted humans. This effect can depend on vision or visual imagery, or alternatively, could be driven by mechanisms independent of visual experience. Here, we show that the actions of reaching in different directions can be reliably decoded from fMRI activity of EVC in congenitally blind humans (both sexes). Thus, neither visual experience nor visual imagery is necessary for EVC to represent action-related information. We also demonstrate that, within EVC of blind humans, the accuracy of reach direction decoding is highest in areas typically representing foveal vision and gradually decreases in areas typically representing peripheral vision. We propose that this might indicate the existence of a predictive, hard-wired mechanism of aligning action and visual spaces. This mechanism might send action-related information primarily to the high-resolution foveal visual areas, which are critical for guiding and online correction of motor actions. Finally, we show that, beyond EVC, the decoding of reach direction in blind humans is most accurate in dorsal stream areas known to be critical for visuo-spatial and visuo-motor integration in the sighted. Thus, these areas can develop space and action representations even in the lifelong absence of vision. Overall, our findings in congenitally blind humans match previous research on the action system in the sighted, and suggest that the development of action representations in the human brain might be largely independent of visual experience.SIGNIFICANCE STATEMENT Early visual cortex (EVC) was traditionally thought to process only visual signals from the retina. Recent studies proved this account incomplete, and showed EVC involvement in many activities not directly related to incoming visual information, such as memory, sound, or action processing. Is EVC involved in these activities because of visual imagery? Here, we show robust reach direction representation in EVC of humans born blind. This demonstrates that EVC can represent actions independently of vision and visual imagery. Beyond EVC, we found that reach direction representation in blind humans is strongest in dorsal brain areas, critical for action processing in the sighted. This suggests that the development of action representations in the human brain is largely independent of visual experience.

The neural correlates of texture perception: A systematic review and activation likelihood estimation meta-analysis of functional magnetic resonance imaging studies

INTRODUCTION

Humans use discriminative touch to perceive texture through dynamic interactions with surfaces, activating low-threshold mechanoreceptors in the skin. It was largely assumed that texture was processed in primary somatosensory regions in the brain; however, imaging studies indicate heterogeneous patterns of brain activity associated with texture processing.

METHODS

To address this, we conducted a coordinate-based activation likelihood estimation meta-analysis of 13 functional magnetic resonance imaging studies (comprising 15 experiments contributing 228 participants and 275 foci) selected by a systematic review.

RESULTS

Concordant activations for texture perception occurred in the left primary somatosensory and motor regions, with bilateral activations in the secondary somatosensory, posterior insula, and premotor and supplementary motor cortices. We also evaluated differences between studies that compared touch processing to non-haptic control (e.g., rest or visual control) or those that used haptic control (e.g., shape or orientation perception) to specifically investigate texture encoding. Studies employing a haptic control revealed concordance for texture processing only in the left secondary somatosensory cortex. Contrast analyses demonstrated greater concordance of activations in the left primary somatosensory regions and inferior parietal cortex for studies with a non-haptic control, compared to experiments accounting for other haptic aspects.

CONCLUSION

These findings suggest that texture processing may recruit higher order integrative structures, and the secondary somatosensory cortex may play a key role in encoding textural properties. The present study provides unique insight into the neural correlates of texture-related processing by assessing the influence of non-textural haptic elements and identifies opportunities for a future research design to understand the neural processing of texture.

Beyond sense-specific processing: decoding texture in the brain from touch and sonified movement

Texture, a fundamental object attribute, is perceived through multisensory information including touch and auditory cues. Coherent perceptions may rely on shared texture representations across different senses in the brain. To test this hypothesis, we delivered haptic textures coupled with a sound synthesizer to generate real-time textural sounds. Participants completed roughness estimation tasks with haptic, auditory, or bimodal cues in an MRI scanner. Somatosensory, auditory, and visual cortices were all activated during haptic and auditory exploration, challenging the traditional view that primary sensory cortices are sense-specific. Furthermore, audio-tactile integration was found in secondary somatosensory (S2) and primary auditory cortices. Multivariate analyses revealed shared spatial activity patterns in primary motor and somatosensory cortices, for discriminating texture across both modalities. This study indicates that primary areas and S2 have a versatile representation of multisensory textures, which has significant implications for how the brain processes multisensory cues to interact more efficiently with our environment.

Tactile estimation of hedonic and sensory properties during active touch: An electroencephalography study

Perceptual judgements about our physical environment are informed by somatosensory information. In real-world exploration, this often involves dynamic hand movements to contact surfaces, termed active touch. The current study investigated cortical oscillatory changes during active exploration to inform the estimation of surface properties and hedonic preferences of two textured stimuli: smooth silk and rough hessian. A purpose-built touch sensor quantified active touch, and oscillatory brain activity was recorded from 129-channel electroencephalography. By fusing these data streams at a single trial level, oscillatory changes within the brain were examined while controlling for objective touch parameters (i.e., friction). Time-frequency analysis was used to quantify changes in cortical oscillatory activity in alpha (8-12 Hz) and beta (16-24 Hz) frequency bands. Results reproduce findings from our lab, whereby active exploration of rough textures increased alpha-band event-related desynchronisation in contralateral sensorimotor areas. Hedonic processing of less preferred textures resulted in an increase in temporoparietal beta-band and frontal alpha-band event-related desynchronisation relative to most preferred textures, suggesting that higher order brain regions are involved in the hedonic processing of texture. Overall, the current study provides novel insight into the neural mechanisms underlying texture perception during active touch and how this process is influenced by cognitive tasks.

Visuo-haptic object perception for robots: an overview

The object perception capabilities of humans are impressive, and this becomes even more evident when trying to develop solutions with a similar proficiency in autonomous robots. While there have been notable advancements in the technologies for artificial vision and touch, the effective integration of these two sensory modalities in robotic applications still needs to be improved, and several open challenges exist. Taking inspiration from how humans combine visual and haptic perception to perceive object properties and drive the execution of manual tasks, this article summarises the current state of the art of visuo-haptic object perception in robots. Firstly, the biological basis of human multimodal object perception is outlined. Then, the latest advances in sensing technologies and data collection strategies for robots are discussed. Next, an overview of the main computational techniques is presented, highlighting the main challenges of multimodal machine learning and presenting a few representative articles in the areas of robotic object recognition, peripersonal space representation and manipulation. Finally, informed by the latest advancements and open challenges, this article outlines promising new research directions.

Distinguishing transient from persistent tactile agnosia after partial anterior circulation infarcts - Behavioral and neuroimaging evidence for white matter disconnection

From a cohort of 36 patients presenting apperceptive tactile agnosia after first cortical ischemic stroke, 14 showed temporary impairment at admission. A previous multi-voxel analysis of the cortical lesions, using as explanatory variable the course of tactile object recognition performance over the recovery period of 9 months, partitioned the cohort into three subgroups. Of the 14 patients constituting two of the subgroups, 7 recovered from their impairment whereas 7 did not. These two subgroups could not be distinguished at admission. The primary aim of the present study is to present two assessments that can do so. The first assessment comprises a pattern of behavioral measures, determined via principal component analysis, encoded in three tests: picking small objects, macrogeometrical discrimination and tactile object recognition. The receiver operating characteristic curve derived from permutation of the behavioral test scores yielded an 80% probability of correct identification of the patient subgroup and an 8% probability for false identification. As done with the permuted scores, the pattern could predict the persistence of affliction of new stroke patients with tactile agnosia. The second predictive assessment extends our previous evaluation of cortical MRI lesion maps to include subcortical regions. Confirming our previous study, the lesions of the persistently impaired subgroup disrupted significantly the anterior arcuatus fasciculus and associated superior longitudinal fasciculus III in the ipsilesional hemisphere, impeding reciprocal information transfer between supramarginal gyrus and both the ventral premotor cortex and Brodmann area 44. Due to the importance of interhemispheric information transfer in tactile agnosia, we performed a supplementary analysis of tactile object recognition scores. It showed that haptic information transfer from the non-affected to the affected hands in the persistent cases partly restored function during the nine months, possibly following restoration of functional interhemispheric haptic information transfer at the border of posterior corpus callosum and splenium. In conclusion, the combined findings of the cortical lesion at subarea PFt of the inferior parietal lobule and the associated subcortical tract lesions permit almost perfect prediction of persistent impairment of tactile object recognition. The study substantiates the need for combined analysis of both cortical lesions and white matter tract disconnections.

Vision of haptics tunes the somatosensory threshold

The interaction between different sensory modalities represents a crucial issue in the neuroscience of consciousness: when the processing of one modality is deficient, the concomitant presentation of stimuli of other spared modalities may sustain the restoring of the damaged sensory functions. In this regard, visual enhancement of touch may represent a viable tool in the rehabilitation from tactile disorders, yet the specific visual features mostly modulating the somatosensory experience remain unsettled. In this study, healthy subjects underwent a tactile detection task during the observation of videos displaying different contents, including static gratings, meaningless motions, natural or point-lights reach-to-grasp-and-manipulate actions. Concurrently, near-threshold stimuli were delivered to the median nerve at different time-points. Subjective report was collected after each trial; the sensory detection rate was computed and compared across video conditions. Our results indicate that the specific presence of haptic contents (i.e., vision of manipulation), either fully displayed or implied by point-lights, magnifies tactile sensitivity. The notion that such stimuli prompt an aware tactile experience opens to novel rehabilitation approaches for tactile consciousness disorders.

Tactile information affects alternating visual percepts during binocular rivalry using naturalistic objects

INTRODUCTION

Past studies have provided evidence that the effects of tactile stimulation on binocular rivalry are mediated by primitive features (orientation and spatial frequency) common in vision and touch. In this study, we examined whether such effects on binocular rivalry can be obtained through the roughness of naturalistic objects. In three experiments, the total dominant time of visual percepts of two objects was measured under binocular rivalry when participants touched one of the objects.

RESULT

In Experiment 1, the total dominant time for the image of artificial turf and bathmat was prolonged by congruent tactile stimulation and shortened by incongruent tactile stimulation. In Experiment 2, we used the same stimuli but rotated their visual images in opposite directions. The dominant time for either image was prolonged by congruent tactile stimulation. In Experiment 3, we used different types of stimuli, smooth marble and rough fabric, and noted significant effects of the congruent and incongruent tactile stimulation on the dominant time of visual percepts.

CONCLUSION

These three experiments demonstrated that visuo-tactile interaction on binocular rivalry can be mediated by roughness.

Altered Spontaneous Brain Activity Patterns of Meibomian Gland Dysfunction in Severely Obese Population Measured Using the Fractional Amplitude of Low-Frequency Fluctuations

OBJECTIVE

Utilizing the fractional amplitude of low-frequency fluctuations (fALFF) technique, this study sought to correlate spontaneous cerebral abnormalities with the clinical manifestations of meibomian gland dysfunction (MGD) in severely obese (SO) population.

SUBJECTS AND METHODS

Twelve MGD patients in SO population (PATs) (4 males and 8 females) and twelve healthy controls (HCs) (6 males and 6 females) matched by gender and age were enrolled. Every participant underwent resting-state functional magnetic resonance imaging (rs-MRI) scanning. Spontaneous cerebral activity alterations were examined using the fALFF method. Receiver operating characteristic (ROC) curves were utilized to classify the medial fALFF values of the PATs and HCs. PATs were also asked to complete anxiety and depression score forms, permitting a correlation analysis.

RESULTS

In contrast with HCs, PATs had prominently increased fALFF values in the left lingual gyrus, the right globus pallidus, the right anterior cingulate and paracingulate gyri and the left middle occipital lobe (P < 0.05), and decreased fALFF values in the right cerebellum, the left fusiform gyrus, the right medial orbitofrontal gyrus, the left triangle inferior frontal gyrus and the left inferior parietal gyrus (P < 0.05). The results of the ROC curve indicated that changes in regional fALFF values might help diagnose MGD in SO population. Moreover, fALFF values in the right cerebellum of PATs were positively correlated with hospital anxiety and depression scores (HADS) (r = 0.723, P = 0.008). The fALFF values in the left triangle inferior frontal gyrus of PAT were negatively correlated with HADS (r = -0.651, P = 0.022).

CONCLUSIONS

Aberrant spontaneous activity was observed in multiple regions of the cerebrum, offering helpful information about the pathology of MGD in SO population. Aberrant fALFF values in these regions likely relates to the latent pathologic mechanisms of anomalous cerebral activities in PATs.

Magnetic resonance-based eye tracking using deep neural networks

Viewing behavior provides a window into many central aspects of human cognition and health, and it is an important variable of interest or confound in many functional magnetic resonance imaging (fMRI) studies. To make eye tracking freely and widely available for MRI research, we developed DeepMReye, a convolutional neural network (CNN) that decodes gaze position from the magnetic resonance signal of the eyeballs. It performs cameraless eye tracking at subimaging temporal resolution in held-out participants with little training data and across a broad range of scanning protocols. Critically, it works even in existing datasets and when the eyes are closed. Decoded eye movements explain network-wide brain activity also in regions not associated with oculomotor function. This work emphasizes the importance of eye tracking for the interpretation of fMRI results and provides an open source software solution that is widely applicable in research and clinical settings.

Haptic object recognition based on shape relates to visual object recognition ability

Visual object recognition depends in large part on a domain-general ability (Richler et al. Psychol Rev 126(2): 226–251, 2019). Given evidence pointing towards shared mechanisms for object perception across vision and touch, we ask whether individual differences in haptic and visual object recognition are related. We use existing validated visual tests to estimate visual object recognition ability and relate it to performance on two novel tests of haptic object recognition ability (n = 66). One test includes complex objects that participants chose to explore with a hand grasp. The other test uses a simpler stimulus set that participants chose to explore with just their fingertips. Only performance on the haptic test with complex stimuli correlated with visual object recognition ability, suggesting a shared source of variance across task structures, stimuli, and modalities. A follow-up study using a visual version of the haptic test with simple stimuli shows a correlation with the original visual tests, suggesting that the limited complexity of the stimuli did not limit correlation with visual object recognition ability. Instead, we propose that the manner of exploration may be a critical factor in whether a haptic test relates to visual object recognition ability. Our results suggest a perceptual ability that spans at least across vision and touch, however, it may not be recruited during just fingertip exploration.

Temporal Dynamics of Brain White Matter Plasticity in Sighted Subjects during Tactile Braille Learning: A Longitudinal Diffusion Tensor Imaging Study

The white matter (WM) architecture of the human brain changes in response to training, though fine-grained temporal characteristics of training-induced white matter plasticity remain unexplored. We investigated white matter microstructural changes using diffusion tensor imaging at five different time points in 26 sighted female adults during 8 months of training on tactile braille reading. Our results show that training-induced white matter plasticity occurs both within and beyond the trained sensory modality, as reflected by fractional anisotropy (FA) increases in somatosensory and visual cortex, respectively. The observed changes followed distinct time courses, with gradual linear FA increase along the training in the somatosensory cortex and sudden visual cortex cross-modal plasticity occurring after braille input became linguistically meaningful. WM changes observed in these areas returned to baseline after the cessation of learning in line with the supply-demand model of plasticity. These results also indicate that the temporal dynamics of microstructural plasticity in different cortical regions might be modulated by the nature of computational demands. We provide additional evidence that observed FA training-induced changes are behaviorally relevant to tactile reading. Together, these results demonstrate that WM plasticity is a highly dynamic process modulated by the introduction of novel experiences.SIGNIFICANCE STATEMENT Throughout the lifetime the human brain is shaped by various experiences. Training-induced reorganization in white matter (WM) microstructure has been reported, but we know little about its temporal dynamics. To fill this gap, we scanned sighted subjects five times during tactile braille reading training. We observed different dynamics of WM plasticity in the somatosensory and visual cortices implicated in braille reading. The former showed a continuous increase in WM tissue anisotropy along with tactile training, while microstructural changes in the latter were observed only after the participants learned to read braille words. Our results confirm the supply-demand model of brain plasticity and provide evidence that WM reorganization depends on distinct computational demands and functional roles of regions involved in the trained skill.

Evaluating fMRI-Based Estimation of Eye Gaze During Naturalistic Viewing

The collection of eye gaze information during functional magnetic resonance imaging (fMRI) is important for monitoring variations in attention and task compliance, particularly for naturalistic viewing paradigms (e.g., movies). However, the complexity and setup requirements of current in-scanner eye tracking solutions can preclude many researchers from accessing such information. Predictive eye estimation regression (PEER) is a previously developed support vector regression-based method for retrospectively estimating eye gaze from the fMRI signal in the eye's orbit using a 1.5-min calibration scan. Here, we provide confirmatory validation of the PEER method's ability to infer eye gaze on a TR-by-TR basis during movie viewing, using simultaneously acquired eye tracking data in five individuals (median angular deviation < 2°). Then, we examine variations in the predictive validity of PEER models across individuals in a subset of data (n = 448) from the Child Mind Institute Healthy Brain Network Biobank, identifying head motion as a primary determinant. Finally, we accurately classify which of the two movies is being watched based on the predicted eye gaze patterns (area under the curve = 0.90 ± 0.02) and map the neural correlates of eye movements derived from PEER. PEER is a freely available and easy-to-use tool for determining eye fixations during naturalistic viewing.

Brain networks underlying the processing of sound symbolism related to softness perception

Unlike the assumption of modern linguistics, there is non-arbitrary association between sound and meaning in sound symbolic words. Neuroimaging studies have suggested the unique contribution of the superior temporal sulcus to the processing of sound symbolism. However, because these findings are limited to the mapping between sound symbolism and visually presented objects, the processing of sound symbolic information may also involve the sensory-modality dependent mechanisms. Here, we conducted a functional magnetic resonance imaging experiment to test whether the brain regions engaged in the tactile processing of object properties are also involved in mapping sound symbolic information with tactually perceived object properties. Thirty-two healthy subjects conducted a matching task in which they judged the congruency between softness perceived by touch and softness associated with sound symbolic words. Congruency effect was observed in the orbitofrontal cortex, inferior frontal gyrus, insula, medial superior frontal gyrus, cingulate gyrus, and cerebellum. This effect in the insula and medial superior frontal gyri was overlapped with softness-related activity that was separately measured in the same subjects in the tactile experiment. These results indicate that the insula and medial superior frontal gyrus play a role in processing sound symbolic information and relating it to the tactile softness information.

Different activation signatures in the primary sensorimotor and higher-level regions for haptic three-dimensional curved surface exploration

Haptic object perception begins with continuous exploratory contact, and the human brain needs to accumulate sensory information continuously over time. However, it is still unclear how the primary sensorimotor cortex (PSC) interacts with these higher-level regions during haptic exploration over time. This functional magnetic resonance imaging (fMRI) study investigates time-dependent haptic object processing by examining brain activity during haptic 3D curve and roughness estimations. For this experiment, we designed sixteen haptic stimuli (4 kinds of curves × 4 varieties of roughness) for the haptic curve and roughness estimation tasks. Twenty participants were asked to move their right index and middle fingers along the surface twice and to estimate one of the two features-roughness or curvature-depending on the task instruction. We found that the brain activity in several higher-level regions (e.g., the bilateral posterior parietal cortex) linearly increased as the number of curves increased during the haptic exploration phase. Surprisingly, we found that the contralateral PSC was parametrically modulated by the number of curves only during the late exploration phase but not during the early exploration phase. In contrast, we found no similar parametric modulation activity patterns during the haptic roughness estimation task in either the contralateral PSC or in higher-level regions. Thus, our findings suggest that haptic 3D object perception is processed across the cortical hierarchy, whereas the contralateral PSC interacts with other higher-level regions across time in a manner that is dependent upon the features of the object.

Evidence of cortical thickness reduction and disconnection in high myopia

High myopia (HM) is associated with impaired long-distance vision. accumulating evidences reported that abnormal visual experience leads to dysfunction in brain activity in HM even corrected. However, whether the long-term of abnormal visual experience lead to neuroanatomical changes remain unknown, the aim at this study is to investigate the alternation of cortical surface thickness in HM patients. 82 patients with HM (HM groups), 57 healthy controls (HC groups) were recruited. All participants underwent high-resolution T1 and resting-state functional magnetic resonance imaging (MRI) scans. The cortical thickness analysis was preformed to investigate the neuroanatomical changes in HM patients using computational anatomy toolbox (CAT 12) toolbox. Compare with HCs, HM patients showed decreased the cortical surface thickness in the left middle occipital gyrus (MOG), left inferior parietal lobule (IPL), right inferior temporal gyrus (ITG), right precuneus, right primary visual area 1 (V1), right superior temporal gyrus (STG), right superior parietal lobule (SPL), right occipital pole, and right the primary motor cortex (M1), and increased to the parietal operculum (OP4) (P < 0.01, FWE-corrected), the mean cortical thickness of right orbitofrontal cortex (OFC), right dorsolateral prefrontal cortex (DLPFC) and right subcallosal cortex showed negatively correlation between clinical variables (axis length (ALM), the average macular thickness (AMT), keratometer (KER) 1, KER2, the mean KER, the mean macular fovea thickness (MFK), the refractive diopter) in HM patients. Our result mainly provided an evidence of cortical thickness reduction and disconnection in visual center and visual processing area, and cortical thickness increase in left multimodal integration region in HM patients. This may provide important significance of the study of the neural mechanism of HM.

The Nature of Haptic Working Memory Capacity and Its Relation to Visual Working Memory

I conducted three experiments to investigate haptic working memory capacity using a haptic change detection task with 2D stimuli. I adopted a single-task paradigm comprising haptic single-feature (orientation or texture) and haptic multifeature (orientation and texture) conditions in Experiment 1 and a dual-task paradigm with a primary haptic orientation or texture change detection task and a concurrent secondary visual shape or colour change detection task in Experiments 2-3. I observed that in the single-task paradigm, haptic change detection capacity was higher for single features than it was for multiple features. In haptic working memory, unlike in visual working memory, features of two different dimensions within an object cannot be integrated. In the dual-task paradigm, interference was observed when the concurrent visual shape change detection task was combined with the haptic orientation change detection task although interference was not observed when the concurrent visual colour change detection task was combined with it. In addition, the concurrent visual shape or colour change detection task did not interfere with the capacity for haptic texture memory, which was higher than that for haptic orientation memory. These findings demonstrate that geometric properties perhaps retained a common storage system shared between haptic working memory and visual working memory; however, haptic texture might be retained in an independent stable storage system that is haptic-specific.

Visual motion information modulates tactile roughness perception

We perceive the roughness of an object through our eyes and hands. Many crossmodal studies have reported that there is no clear visuo-tactile interaction in roughness perception using static visual cues. One exception is that the visual observation of task-irrelevant hand movements, not the texture of task-relevant objects, can enhance the performance of tactile roughness discrimination. Our study investigated whether task-irrelevant visual motion without either object roughness or bodily cues can influence tactile roughness perception. Participants were asked to touch abrasive papers while moving their hand laterally and viewing moving or static sine wave gratings without being able to see their hand, and to estimate the roughness magnitude of the tactile stimuli. Moving gratings with a low spatial frequency induced smoother roughness perceptions than static visual stimuli when the visual grating moved in the direction opposite the hand movements. The effects of visual motion did not appear when the visual stimuli had a high spatial frequency or when the participants touched the tactile stimuli passively. These results indicate that simple task-irrelevant visual movement without object roughness or bodily cues can modulate tactile roughness perception with active body movements in a spatial-frequency-selective manner.

Shitsukan - the Multisensory Perception of Quality

We often estimate, or perceive, the quality of materials, surfaces, and objects, what the Japanese refer to as 'shitsukan', by means of several of our senses. The majority of the literature on shitsukan perception has, though, tended to focus on the unimodal visual evaluation of stimulus properties. In part, this presumably reflects the widespread hegemony of the visual in the modern era and, in part, is a result of the growing interest, not to mention the impressive advances, in digital rendering amongst the computer graphics community. Nevertheless, regardless of such an oculocentric bias in so much of the empirical literature, it is important to note that several other senses often do contribute to the impression of the material quality of surfaces, materials, and objects as experienced in the real world, rather than just in virtual reality. Understanding the multisensory contributions to the perception of material quality, especially when combined with computational and neural data, is likely to have implications for a number of fields of basic research as well as being applicable to emerging domains such as, for example, multisensory augmented retail, not to mention multisensory packaging design.

Do Process-1 simulations generate the epistemic feelings that drive Process-2 decision making?

We apply previously developed Chu space and Channel Theory methods, focusing on the construction of Cone-Cocone Diagrams (CCCDs), to study the role of epistemic feelings, particularly feelings of confidence, in dual process models of problem solving. We specifically consider "Bayesian brain" models of probabilistic inference within a global neuronal workspace architecture. We develop a formal representation of Process-1 problem solving in which a solution is reached if and only if a CCCD is completed. We show that in this representation, Process-2 problem solving can be represented as multiply iterated Process-1 problem solving and has the same formal solution conditions. We then model the generation of explicit, reportable subjective probabilities from implicit, experienced confidence as a simulation-based, reverse engineering process and show that this process can also be modeled as a CCCD construction.

Multi-Level Clustering of Dynamic Directional Brain Network Patterns and Their Behavioral Relevance

Dynamic functional connectivity (DFC) obtained from resting state functional magnetic resonance imaging (fMRI) data has been shown to provide novel insights into brain function which may be obscured by static functional connectivity (SFC). Further, DFC, and by implication how different brain regions may engage or disengage with each other over time, has been shown to be behaviorally relevant and more predictive than SFC of behavioral performance and/or diagnostic status. DFC is not a directional entity and may capture neural synchronization. However, directional interactions between different brain regions is another putative mechanism by which neural populations communicate. Accordingly, static effective connectivity (SEC) has been explored as a means of characterizing such directional interactions. But investigation of its dynamic counterpart, i.e., dynamic effective connectivity (DEC), is still in its infancy. Of particular note are methodological insufficiencies in identifying DEC configurations that are reproducible across time and subjects as well as a lack of understanding of the behavioral relevance of DEC obtained from resting state fMRI. In order to address these issues, we employed a dynamic multivariate autoregressive (MVAR) model to estimate DEC. The method was first validated using simulations and then applied to resting state fMRI data obtained in-house (N = 21), wherein we performed dynamic clustering of DEC matrices across multiple levels [using adaptive evolutionary clustering (AEC)] – spatial location, time, and subjects. We observed a small number of directional brain network configurations alternating between each other over time in a quasi-stable manner akin to brain microstates. The dominant and consistent DEC network patterns involved several regions including inferior and mid temporal cortex, motor and parietal cortex, occipital cortex, as well as part of frontal cortex. The functional relevance of these DEC states were determined using meta-analyses and pertained mainly to memory and emotion, but also involved execution and language. Finally, a larger cohort of resting-state fMRI and behavioral data from the Human Connectome Project (HCP) (N = 232, Q1–Q3 release) was used to demonstrate that metrics derived from DEC can explain larger variance in 70 behaviors across different domains (alertness, cognition, emotion, and personality traits) compared to SEC in healthy individuals.

Deterioration from healthy to mild cognitive impairment and Alzheimer's disease mirrored in corresponding loss of centrality in directed brain networks

OBJECTIVE

It is important to identify brain-based biomarkers that progressively deteriorate from healthy to mild cognitive impairment (MCI) to Alzheimer's disease (AD). Cortical thickness, amyloid-ß deposition, and graph measures derived from functional connectivity (FC) networks obtained using functional MRI (fMRI) have been previously identified as potential biomarkers. Specifically, in the latter case, betweenness centrality (BC), a nodal graph measure quantifying information flow, is reduced in both AD and MCI. However, all such reports have utilized BC calculated from undirected networks that characterize synchronization rather than information flow, which is better characterized using directed networks.

METHODS

Therefore, we estimated BC from directed networks using Granger causality (GC) on resting-state fMRI data (N = 132) to compare the following populations (p < 0.05, FDR corrected for multiple comparisons): normal control (NC), early MCI (EMCI), late MCI (LMCI) and AD. We used an additional metric called middleman power (MP), which not only characterizes nodal information flow as in BC, but also measures nodal power critical for information flow in the entire network.

RESULTS

MP detected more brain regions than BC that progressively deteriorated from NC to EMCI to LMCI to AD, as well as exhibited significant associations with behavioral measures. Additionally, graph measures obtained from conventional FC networks could not identify a single node, underscoring the relevance of GC.

CONCLUSION

Our findings demonstrate the superiority of MP over BC as well as GC over FC in our case. MP obtained from GC networks could serve as a potential biomarker for progressive deterioration of MCI and AD.

Effective connectivity within a triple network brain system discriminates schizophrenia spectrum disorders from psychotic bipolar disorder at the single-subject level

OBJECTIVE

Schizophrenia spectrum disorders (SSD) and psychotic bipolar disorder share a number of genetic and neurobiological features, despite a divergence in clinical course and outcome trajectories. We studied the diagnostic classification potential that can be achieved on the basis of the structure and connectivity within a triple network system (the default mode, salience and central executive network) in patients with SSD and psychotic bipolar disorder.

METHODS

Directed static connectivity and its dynamic variance was estimated among 8 nodes of the three large-scale networks. Multivariate autoregressive models of deconvolved resting state functional magnetic resonance imaging time series were obtained from 57 patients (38 with SSD and 19 with bipolar disorder and psychosis). We used 2/3 of the patients for training and validation of the classifier and the remaining 1/3 as an independent hold-out test data for performance estimation.

RESULTS

A high level of discrimination between bipolar disorder with psychosis and SSD (combined balanced accuracy = 96.2%; class accuracies 100% for bipolar and 92.3% for SSD) was achieved when effective connectivity and morphometry of the triple network nodes was combined with symptom scores. Patients with SSD were discriminated from patients with bipolar disorder and psychosis as showing higher clinical severity of disorganization and higher variability in the effective connectivity between salience and executive networks.

CONCLUSIONS

Our results support the view that the study of network-level connectivity patterns can not only clarify the pathophysiology of SSD but also provide a measure of excellent clinical utility to identify discrete diagnostic/prognostic groups among individuals with psychosis.

Active touch facilitates object size perception in children but not adults: A multisensory event related potential study

In order to increase perceptual precision the adult brain dynamically combines redundant information from different senses depending on their reliability. During object size estimation, for example, visual, auditory and haptic information can be integrated to increase the precision of the final size estimate. Young children, however, do not integrate sensory information optimally and instead rely on active touch. Whether this early haptic dominance is reflected in age-related differences in neural mechanisms and whether it is driven by changes in bottom-up perceptual or top-down attentional processes has not yet been investigated. Here, we recorded event-related-potentials from a group of adults and children aged 5-7 years during an object size perception task using auditory, visual and haptic information. Multisensory information was presented either congruently (conveying the same information) or incongruently (conflicting information). No behavioral responses were required from participants. When haptic size information was available via actively tapping the objects, response amplitudes in the mid-parietal area were significantly reduced by information congruency in children but not in adults between 190 ms-250 ms and 310 ms-370 ms. These findings indicate that during object size perception only children's brain activity is modulated by active touch supporting a neural maturational shift from sensory dominance in early childhood to optimal multisensory benefit in adulthood.

Functional hierarchy for tactile processing in the visual cortex of sighted adults

Perception via different sensory modalities was traditionally believed to be supported by largely separate brain systems. However, a growing number of studies demonstrate that the visual cortices of typical, sighted adults are involved in tactile and auditory perceptual processing. Here, we investigated the spatiotemporal dynamics of the visual cortex's involvement in a complex tactile task: Braille letter recognition. Sighted subjects underwent Braille training and then participated in a transcranial magnetic stimulation (TMS) study in which they tactually identified single Braille letters. During this task, TMS was applied to their left early visual cortex, visual word form area (VWFA), and left early somatosensory cortex at five time windows from 20 to 520 ms following the Braille letter presentation's onset. The subjects' response accuracy decreased when TMS was applied to the early visual cortex at the 120-220 ms time window and when TMS was applied to the VWFA at the 320-420 ms time window. Stimulation of the early somatosensory cortex did not have a time-specific effect on the accuracy of the subjects' Braille letter recognition, but rather caused a general slowdown during this task. Our results indicate that the involvement of sighted people's visual cortices in tactile perception respects the canonical visual hierarchy-the early tactile processing stages involve the early visual cortex, whereas more advanced tactile computations involve high-level visual areas. Our findings are compatible with the metamodal account of brain organization and suggest that the whole visual cortex may potentially support spatial perception in a task-specific, sensory-independent manner.

Hand-Feel Touch Cues and Their Influences on Consumer Perception and Behavior with Respect to Food Products: A Review

There has been a great deal of research investigating intrinsic/extrinsic cues and their influences on consumer perception and purchasing decisions at points of sale, product usage, and consumption. Consumers create expectations toward a food product through sensory information extracted from its surface (intrinsic cues) or packaging (extrinsic cues) at retail stores. Packaging is one of the important extrinsic cues that can modulate consumer perception, liking, and decision making of a product. For example, handling a product packaging during consumption, even just touching the packaging while opening or holding it during consumption, may result in a consumer expectation of the package content. Although hand-feel touch cues are an integral part of the food consumption experience, as can be observed in such an instance, little has been known about their influences on consumer perception, acceptability, and purchase behavior of food products. This review therefore provided a better understanding about hand-feel touch cues and their influences in the context of food and beverage experience with a focus on (1) an overview of touch as a sensory modality, (2) factors influencing hand-feel perception, (3) influences of hand-feel touch cues on the perception of other sensory modalities, and (4) the effects of hand-feel touch cues on emotional responses and purchase behavior.

Brain networks underlying tactile softness perception: A functional magnetic resonance imaging study

Humans are adept at perceiving physical properties of an object through touch. Tangible object properties can be categorized into two types: macro-spatial properties, including shape and orientation; and material properties, such as roughness, softness, and temperature. Previous neuroimaging studies have shown that roughness and temperature are extracted at nodes of a network, such as that involving the parietal operculum and insula, which is different from the network engaged in processing macro-spatial properties. However, it is unclear whether other perceptual dimensions pertaining to material properties engage the same regions. Here, we conducted a functional magnetic resonance imaging study to test whether the parietal operculum and insula were involved in extracting tactually-perceived softness magnitude. Fifty-six healthy right-handed participants estimated perceived softness magnitude using their right middle finger. We presented three stimuli that had the same shape but different compliances. The force applied to the finger was manipulated at two levels. Classical mass-univariate analysis showed that activity in the parietal operculum, insula, and medial prefrontal cortex was positively associated with perceived softness magnitude, regardless of the applied force. Softness-related activity was stronger in the ventral striatum in the high-force condition than in the low-force condition. The multivariate voxel pattern analysis showed higher accuracy than chance levels and control regions in the parietal operculum/insula, postcentral gyrus, posterior parietal lobule, and middle occipital gyrus. These results indicate that a distributed set of the brain regions, including the parietal operculum and insula, is involved in representing perceived softness.

Mental Rotation of Digitally-Rendered Haptic Objects

Sensory substitution is an effective means to rehabilitate many visual functions after visual impairment or blindness. Tactile information, for example, is particularly useful for functions such as reading, mental rotation, shape recognition, or exploration of space. Extant haptic technologies typically rely on real physical objects or pneumatically driven renderings and thus provide a limited library of stimuli to users. New developments in digital haptic technologies now make it possible to actively simulate an unprecedented range of tactile sensations. We provide a proof-of-concept for a new type of technology (hereafter haptic tablet) that renders haptic feedback by modulating the friction of a flat screen through ultrasonic vibrations of varying shapes to create the sensation of texture when the screen is actively explored. We reasoned that participants should be able to create mental representations of letters presented in normal and mirror-reversed haptic form without the use of any visual information and to manipulate such representations in a mental rotation task. Healthy sighted, blindfolded volunteers were trained to discriminate between two letters (either L and P, or F and G; counterbalanced across participants) on a haptic tablet. They then tactually explored all four letters in normal or mirror-reversed form at different rotations (0°, 90°, 180°, and 270°) and indicated letter form (i.e., normal or mirror-reversed) by pressing one of two mouse buttons. We observed the typical effect of rotation angle on object discrimination performance (i.e., greater deviation from 0° resulted in worse performance) for trained letters, consistent with mental rotation of these haptically-rendered objects. We likewise observed generally slower and less accurate performance with mirror-reversed compared to prototypically oriented stimuli. Our findings extend existing research in multisensory object recognition by indicating that a new technology simulating active haptic feedback can support the generation and spatial manipulation of mental representations of objects. Thus, such haptic tablets can offer a new avenue to mitigate visual impairments and train skills dependent on mental object-based representations and their spatial manipulation.

Accounting for Non-Gaussian Sources of Spatial Correlation in Parametric Functional Magnetic Resonance Imaging Paradigms I: Revisiting Cluster-Based Inferences

In a recent study, Eklund et al. employed resting-state functional magnetic resonance imaging data as a surrogate for null functional magnetic resonance imaging (fMRI) datasets and posited that cluster-wise family-wise error (FWE) rate-corrected inferences made by using parametric statistical methods in fMRI studies over the past two decades may have been invalid, particularly for cluster defining thresholds less stringent than p < 0.001; this was principally because the spatial autocorrelation functions (sACF) of fMRI data had been modeled incorrectly to follow a Gaussian form, whereas empirical data suggested otherwise. Here, we show that accounting for non-Gaussian signal components such as those arising from resting-state neural activity as well as physiological responses and motion artifacts in the null fMRI datasets yields first- and second-level general linear model analysis residuals with nearly uniform and Gaussian sACF. Further comparison with nonparametric permutation tests indicates that cluster-based FWE corrected inferences made with Gaussian spatial noise approximations are valid.

Altered Effective Connectivity in the Default Network of the Brains of First-Episode, Drug-Naïve Schizophrenia Patients With Auditory Verbal Hallucinations

Although the default mode network (DMN) is known to be abnormal in schizophrenia (SZ) patients with auditory verbal hallucinations (AVHs), it is still unclear whether AVHs that occur in SZ are associated with certain information flow in the DMN. This study collected resting-state functional magnetic resonance imaging data from 28 first-episode, drug-naïve SZ patients with AVHs, 20 SZ patients without AVHs, and 38 healthy controls. We used Granger causality analysis (GCA) to examine effective connectivity (EC) of two hub regions [posterior cingulate cortex (PCC) and anteromedial prefrontal cortex (aMPFC)] within the DMN. We used two-sample t-tests to compare the difference in EC between the two patient groups, and used Spearman correlation analysis to characterize the relationship between imaging findings and clinical assessments. The GCA revealed that, compared with the non-AVHs group, EC decreased from aMPFC to left inferior temporal gyrus (ITG) and from PCC to left cerebellum posterior lobe, ITG, and right middle frontal gyrus in SZ patients with AVHs. We also found significant correlations between clinical assessments and mean strengths of connectivity from aMPFC to left ITG and from PCC to left ITG. Moreover, receiver operating characteristic analysis revealed that the above-mentioned effective connectivities had a diagnostic value for distinguishing SZ patients with AVHs from non-AVHs patients. These findings suggest that AVHs in SZ patients may be associated with the aberrant information flows of the DMN, and the left ITG may probably serve as a potential biomarker for the neural mechanisms underlying AVHs in SZ patients.

Neural Evidence of Hierarchical Cognitive Control during Haptic Processing: An fMRI Study

Interacting with our immediate surroundings requires constant manipulation of objects. Dexterous manipulation depends on comparison between actual and predicted sensory input, with these predictions calculated by means of lower- and higher-order corollary discharge signals. However, there is still scarce knowledge about the hierarchy in the neural architecture supporting haptic monitoring during manipulation. The present study aimed to assess this issue focusing on the cross talk between lower-order sensory and higher-order associative regions. We used functional magnetic resonance imaging in humans during a haptic discrimination task in which participants had to judge whether a touched shape or texture corresponded to an expected stimulus whose name was previously presented. Specialized haptic regions identified with an independent localizer task did not differ between expected and unexpected conditions, suggesting their lack of involvement in tactile monitoring. When presented stimuli did not match previous expectations, the left supramarginal gyrus (SMG), middle temporal, and medial prefrontal cortices were activated regardless of the nature of the haptic mismatch (shape/texture). The left primary somatosensory area (SI) responded differently to unexpected shapes and textures in line with a specialized detection of haptic mismatch. Importantly, connectivity analyses revealed that the left SMG and SI were more functionally coupled during unexpected trials, emphasizing their interaction. The results point for the first time to a hierarchical organization in the neural substrates underlying haptic monitoring during manipulation with the SMG as a higher-order hub comparing actual and predicted somatosensory input, and SI as a lower-order site involved in the detection of more specialized haptic mismatch.

Directional brain networks underlying OM chanting

OM chanting is an ancient technique of Indian meditation. OM chanting is associated with an experience of relaxation, changes in autonomic balance and deactivation of limbic brain regions. While functional localization is important, how brain regions interact with each other has been shown to underlie various brain functions. Therefore, in this study, we tested the hypothesis that there is reduced communication between deactivated regions during OM chanting. In order to do so, we employed multivariate autoregressive model (MVAR) based Granger causality to obtain directional connectivity between deactivated regions. fMRI scans of 12 right handed healthy volunteers (9 Men) from a previously published study was used in which participants performed OM chanting and a control condition in a block design. We found that outputs from insula, anterior cingulate and orbitofrontal cortices were significantly reduced in OM condition. Of interest is the reduction of outputs from these regions to limbic area amygdala. Modulation of brain regions involved in emotion processing and implicated in major depressive disorder (MDD) raises a potential possibility of OM chanting in the treatment of MDD.

Neural Mechanisms of Material Perception: Quest on Shitsukan

In recent years, a growing body of research has addressed the nature and mechanism of material perception. Material perception entails perceiving and recognizing a material, surface quality or internal state of an object based on sensory stimuli such as visual, tactile, and/or auditory sensations. This process is ongoing in every aspect of daily life. We can, for example, easily distinguish whether an object is made of wood or metal, or whether a surface is rough or smooth. Judging whether the ground is wet or dry or whether a fish is fresh also involves material perception. Information obtained through material perception can be used to govern actions toward objects and to make decisions about whether to approach an object or avoid it. Because the physical processes leading to sensory signals related to material perception is complicated, it has been difficult to manipulate experimental stimuli in a rigorous manner. However, that situation is now changing thanks to advances in technology and knowledge in related fields. In this article, we will review what is currently known about the neural mechanisms responsible for material perception. We will show that cortical areas in the ventral visual pathway are strongly involved in material perception. Our main focus is on vision, but every sensory modality is involved in material perception. Information obtained through different sensory modalities is closely linked in material perception. Such cross-modal processing is another important feature of material perception, and will also be covered in this review.

Investigating the Correspondence of Clinical Diagnostic Grouping With Underlying Neurobiological and Phenotypic Clusters Using Unsupervised Machine Learning

Many brain-based disorders are traditionally diagnosed based on clinical interviews and behavioral assessments, which are recognized to be largely imperfect. Therefore, it is necessary to establish neuroimaging-based biomarkers to improve diagnostic precision. Resting-state functional magnetic resonance imaging (rs-fMRI) is a promising technique for the characterization and classification of varying disorders. However, most of these classification methods are supervised, i.e., they require a priori clinical labels to guide classification. In this study, we adopted various unsupervised clustering methods using static and dynamic rs-fMRI connectivity measures to investigate whether the clinical diagnostic grouping of different disorders is grounded in underlying neurobiological and phenotypic clusters. In order to do so, we derived a general analysis pipeline for identifying different brain-based disorders using genetic algorithm-based feature selection, and unsupervised clustering methods on four different datasets; three of them-ADNI, ADHD-200, and ABIDE-which are publicly available, and a fourth one-PTSD and PCS-which was acquired in-house. Using these datasets, the effectiveness of the proposed pipeline was verified on different disorders: Attention Deficit Hyperactivity Disorder (ADHD), Alzheimer's Disease (AD), Autism Spectrum Disorder (ASD), Post-Traumatic Stress Disorder (PTSD), and Post-Concussion Syndrome (PCS). For ADHD and AD, highest similarity was achieved between connectivity and phenotypic clusters, whereas for ASD and PTSD/PCS, highest similarity was achieved between connectivity and clinical diagnostic clusters. For multi-site data (ABIDE and ADHD-200), we report site-specific results. We also reported the effect of elimination of outlier subjects for all four datasets. Overall, our results suggest that neurobiological and phenotypic biomarkers could potentially be used as an aid by the clinician, in additional to currently available clinical diagnostic standards, to improve diagnostic precision. Data and source code used in this work is publicly available at https://github.com/xinyuzhao/identification-of-brain-based-disorders.git.

Brain networks underlying conscious tactile perception of textures as revealed using the velvet hand illusion

Humans are adept at perceiving textures through touch. Previous neuroimaging studies have identified a distributed network of brain regions involved in the tactile perception of texture. However, it remains unclear how nodes in this network contribute to the tactile awareness of texture. To examine the hypothesis that such awareness involves the interaction of the primary somatosensory cortex with higher order cortices, we conducted a functional magnetic resonance imaging (fMRI) study utilizing the velvet hand illusion, in which an illusory velvet-like surface is perceived between the hands. Healthy participants were subjected to a strong illusion, a weak illusion, and tactile perception of real velvet. The strong illusion induced greater activation in the primary somatosensory cortex (S1) than the weak illusion, and increases in such activation were positively correlated with the strength of the illusion. Furthermore, both actual and illusory perception of velvet induced common activation in S1. Psychophysiological interaction (PPI) analysis revealed that the strength of the illusion modulated the functional connectivity of S1 with each of the following regions: the parietal operculum, superior parietal lobule, precentral gyrus, insula, and cerebellum. The present results indicate that S1 is associated with the conscious tactile perception of textures, which may be achieved via interactions with higher order somatosensory areas.

Tactile-to-Visual Cross-Modal Transfer of Texture Categorisation Following Training: An fMRI Study

We investigated the neural underpinnings of texture categorisation using exemplars that were previously learned either within modalities (visual training and visual test) or across modalities (tactile training and visual test). Previous models of learning suggest a decrease in activation in brain regions that are typically involved in cognitive control during task acquisition, but a concomitant increase in activation in brain regions associated with the representation of the acquired information. In our study, participants were required to learn to categorise fabrics of different textures as either natural or synthetic. Training occurred over several sessions, with each fabric presented either visually or through touch to a participant. Pre- and post-training tests, in which participants categorised visual images only of the fabrics, were conducted during a functional magnetic resonance imaging (fMRI) scan. Consistent with previous research on cognitive processes involved in task acquisition, we found that categorisation training was associated with a decrease in activation in brain regions associated with cognitive systems involved in learning, including the superior parietal cortex, dorsal anterior cingulate cortex (dACC), and the right dorsolateral prefrontal cortex (DLFC). Moreover, these decreases were independent of training modality. In contrast, we found greater activation to visual textures in a region within the left medial occipital cortex (MOC) following training. There was no overall evidence of an effect of training modality in the main analyses, with texture-specific regional changes associated with both within- (visual) and cross- (touch) modal training. However, further analyses suggested that, unlike categorisation performance following within-modal training, crossmodal training was associated with bilateral activation of the MOC. Our results support previous evidence for a multisensory representation of texture within early visual regions of the cortex and provide insight into how multisensory categories are formed in the brain.

Force feedback delay affects perception of stiffness but not action, and the effect depends on the hand used but not on the handedness

Interaction with an object often requires the estimation of its mechanical properties. We examined whether the hand that is used to interact with the object and their handedness affected people's estimation of these properties using stiffness estimation as a test case. We recorded participants' responses on a stiffness discrimination of a virtual elastic force field and the grip force applied on the robotic device during the interaction. In half of the trials, the robotic device delayed the participants' force feedback. Consistent with previous studies, delayed force feedback biased the perceived stiffness of the force field. Interestingly, in both left-handed and right-handed participants, for the delayed force field, there was even less perceived stiffness when participants used their left hand than their right hand. This result supports the idea that haptic processing is affected by laterality in the brain, not by handedness. Consistent with previous studies, participants adjusted their applied grip force according to the correct size and timing of the load force regardless of the hand that was used, the handedness, or the delay. This suggests that in all of these conditions, participants were able to form an accurate internal representation of the anticipated trajectory of the load force (size and timing) and that this representation was used for accurate control of grip force independently of the perceptual bias. Thus these results provide additional evidence for the dissociation between action and perception in the processing of delayed information. NEW & NOTEWORTHY Introducing delay to force feedback during interaction with an elastic force field biases the perceived stiffness of the force field. We show that this bias depends on the hand that was used for probing but not on handedness. At the same time, both left-handed and right-handed participants adjusted their applied grip force while using either their left or right hands in anticipation of the correct magnitude and timing despite the delay in load force.

Multiple time courses of somatosensory responses in human cortex

Here we show how anatomical and functional data recorded from patients undergoing stereo-EEG can be used to decompose the cortical processing following nerve stimulation in different stages characterized by specific topography and time course. Tibial, median and trigeminal nerves were stimulated in 96 patients, and the increase in gamma power was evaluated over 11878 cortical sites. All three nerve datasets exhibited similar clusters of time courses: phasic, delayed/prolonged and tonic, which differed in topography, temporal organization and degree of spatial overlap. Strong phasic responses of the three nerves followed the classical somatotopic organization of SI, with no overlap in either time or space. Delayed responses presented overlaps between pairs of body parts in both time and space, and were confined to the dorsal motor cortices. Finally, tonic responses occurred in the perisylvian region including posterior insular cortex and were evoked by the stimulation of all three nerves, lacking any spatial and temporal specificity. These data indicate that the somatosensory processing following nerve stimulation is a multi-stage hierarchical process common to all three nerves, with the different stages likely subserving different functions. While phasic responses represent the neural basis of tactile perception, multi-nerve tonic responses may represent the neural signature of processes sustaining the capacity to become aware of tactile stimuli.

Evaluating Integration Strategies for Visuo-Haptic Object Recognition

In computational systems for visuo-haptic object recognition, vision and haptics are often modeled as separate processes. But this is far from what really happens in the human brain, where cross- as well as multimodal interactions take place between the two sensory modalities. Generally, three main principles can be identified as underlying the processing of the visual and haptic object-related stimuli in the brain: (1) hierarchical processing, (2) the divergence of the processing onto substreams for object shape and material perception, and (3) the experience-driven self-organization of the integratory neural circuits. The question arises whether an object recognition system can benefit in terms of performance from adopting these brain-inspired processing principles for the integration of the visual and haptic inputs. To address this, we compare the integration strategy that incorporates all three principles to the two commonly used integration strategies in the literature. We collected data with a NAO robot enhanced with inexpensive contact microphones as tactile sensors. The results of our experiments involving every-day objects indicate that (1) the contact microphones are a good alternative to capturing tactile information and that (2) organizing the processing of the visual and haptic inputs hierarchically and in two pre-processing streams is helpful performance-wise. Nevertheless, further research is needed to effectively quantify the role of each identified principle by itself as well as in combination with others.

An open resource for transdiagnostic research in pediatric mental health and learning disorders

Technological and methodological innovations are equipping researchers with unprecedented capabilities for detecting and characterizing pathologic processes in the developing human brain. As a result, ambitions to achieve clinically useful tools to assist in the diagnosis and management of mental health and learning disorders are gaining momentum. To this end, it is critical to accrue large-scale multimodal datasets that capture a broad range of commonly encountered clinical psychopathology. The Child Mind Institute has launched the Healthy Brain Network (HBN), an ongoing initiative focused on creating and sharing a biobank of data from 10,000 New York area participants (ages 5–21). The HBN Biobank houses data about psychiatric, behavioral, cognitive, and lifestyle phenotypes, as well as multimodal brain imaging (resting and naturalistic viewing fMRI, diffusion MRI, morphometric MRI), electroencephalography, eye-tracking, voice and video recordings, genetics and actigraphy. Here, we present the rationale, design and implementation of HBN protocols. We describe the first data release (n=664) and the potential of the biobank to advance related areas (e.g., biophysical modeling, voice analysis).

Structural reorganization of the early visual cortex following Braille training in sighted adults

Training can induce cross-modal plasticity in the human cortex. A well-known example of this phenomenon is the recruitment of visual areas for tactile and auditory processing. It remains unclear to what extent such plasticity is associated with changes in anatomy. Here we enrolled 29 sighted adults into a nine-month tactile Braille-reading training, and used voxel-based morphometry and diffusion tensor imaging to describe the resulting anatomical changes. In addition, we collected resting-state fMRI data to relate these changes to functional connectivity between visual and somatosensory-motor cortices. Following Braille-training, we observed substantial grey and white matter reorganization in the anterior part of early visual cortex (peripheral visual field). Moreover, relative to its posterior, foveal part, the peripheral representation of early visual cortex had stronger functional connections to somatosensory and motor cortices even before the onset of training. Previous studies show that the early visual cortex can be functionally recruited for tactile discrimination, including recognition of Braille characters. Our results demonstrate that reorganization in this region induced by tactile training can also be anatomical. This change most likely reflects a strengthening of existing connectivity between the peripheral visual cortex and somatosensory cortices, which suggests a putative mechanism for cross-modal recruitment of visual areas.

Central attention is serial, but midlevel and peripheral attention are parallel-A hypothesis

In this brief review, we argue that attention operates along a hierarchy from peripheral through central mechanisms. We further argue that these mechanisms are distinguished not just by their functional roles in cognition, but also by a distinction between serial mechanisms (associated with central attention) and parallel mechanisms (associated with midlevel and peripheral attention). In particular, we suggest that peripheral attentional deployments in distinct representational systems may be maintained simultaneously with little or no interference, but that the serial nature of central attention means that even tasks that largely rely on distinct representational systems will come into conflict when central attention is demanded. We go on to review both the behavioral and neural evidence for this prediction. We conclude that even though the existing evidence mostly favors our account of serial central and parallel noncentral attention, we know of no experiment that has conclusively borne out these claims. As such, this article offers a framework of attentional mechanisms that will aid in guiding future research on this topic.

Investigating Focal Connectivity Deficits in Alzheimer's Disease Using Directional Brain Networks Derived from Resting-State fMRI

Connectivity analysis of resting-state fMRI has been widely used to identify biomarkers of Alzheimer's disease (AD) based on brain network aberrations. However, it is not straightforward to interpret such connectivity results since our understanding of brain functioning relies on regional properties (activations and morphometric changes) more than connections. Further, from an interventional standpoint, it is easier to modulate the activity of regions (using brain stimulation, neurofeedback, etc.) rather than connections. Therefore, we employed a novel approach for identifying focal directed connectivity deficits in AD compared to healthy controls. In brief, we present a model of directed connectivity (using Granger causality) that characterizes the coupling among different regions in healthy controls and Alzheimer's disease. We then characterized group differences using a (between-subject) generative model of pathology, which generates latent connectivity variables that best explain the (within-subject) directed connectivity. Crucially, our generative model at the second (between-subject) level explains connectivity in terms of local or regionally specific abnormalities. This allows one to explain disconnections among multiple regions in terms of regionally specific pathology; thereby offering a target for therapeutic intervention. Two foci were identified, locus coeruleus in the brain stem and right orbitofrontal cortex. Corresponding disrupted connectivity network associated with the foci showed that the brainstem is the critical focus of disruption in AD. We further partitioned the aberrant connectomic network into four unique sub-networks, which likely leads to symptoms commonly observed in AD. Our findings suggest that fMRI studies of AD, which have been largely cortico-centric, could in future investigate the role of brain stem in AD.

Dynamic brain connectivity is a better predictor of PTSD than static connectivity

Using resting-state functional magnetic resonance imaging, we test the hypothesis that subjects with post-traumatic stress disorder (PTSD) are characterized by reduced temporal variability of brain connectivity compared to matched healthy controls. Specifically, we test whether PTSD is characterized by elevated static connectivity, coupled with decreased temporal variability of those connections, with the latter providing greater sensitivity toward the pathology than the former. Static functional connectivity (FC; nondirectional zero-lag correlation) and static effective connectivity (EC; directional time-lagged relationships) were obtained over the entire brain using conventional models. Dynamic FC and dynamic EC were estimated by letting the conventional models to vary as a function of time. Statistical separation and discriminability of these metrics between the groups and their ability to accurately predict the diagnostic label of a novel subject were ascertained using separate support vector machine classifiers. Our findings support our hypothesis that PTSD subjects have stronger static connectivity, but reduced temporal variability of connectivity. Further, machine learning classification accuracy obtained with dynamic FC and dynamic EC was significantly higher than that obtained with static FC and static EC, respectively. Furthermore, results also indicate that the ease with which brain regions engage or disengage with other regions may be more sensitive to underlying pathology than the strength with which they are engaged. Future studies must examine whether this is true only in the case of PTSD or is a general organizing principle in the human brain. Hum Brain Mapp 38:4479-4496, 2017. © 2017 Wiley Periodicals, Inc.

Intermodal Attention Shifts in Multimodal Working Memory

Attention maintains task-relevant information in working memory (WM) in an active state. We investigated whether the attention-based maintenance of stimulus representations that were encoded through different modalities is flexibly controlled by top–down mechanisms that depend on behavioral goals. Distinct components of the ERP reflect the maintenance of tactile and visual information in WM. We concurrently measured tactile (tCDA) and visual contralateral delay activity (CDA) to track the attentional activation of tactile and visual information during multimodal WM. Participants simultaneously received tactile and visual sample stimuli on the left and right sides and memorized all stimuli on one task-relevant side. After 500 msec, an auditory retrocue indicated whether the sample set's tactile or visual content had to be compared with a subsequent test stimulus set. tCDA and CDA components that emerged simultaneously during the encoding phase were consistently reduced after retrocues that marked the corresponding (tactile or visual) modality as task-irrelevant. The absolute size of cue-dependent modulations was similar for the tCDA/CDA components and did not depend on the number of tactile/visual stimuli that were initially encoded into WM. Our results suggest that modality-specific maintenance processes in sensory brain regions are flexibly modulated by top–down influences that optimize multimodal WM representations for behavioral goals.