Unraveling the spatiotemporal brain dynamics during a simulated reach-to-eat task

Phase-encoded fMRI tracks down brainstorms of natural language processing with subsecond precision

Natural language processing unfolds information overtime as spatially separated, multimodal, and interconnected neural processes. Existing noninvasive subtraction-based neuroimaging techniques cannot simultaneously achieve the spatial and temporal resolutions required to visualize ongoing information flows across the whole brain. Here we have developed rapid phase-encoded designs to fully exploit the temporal information latent in functional magnetic resonance imaging data, as well as overcoming scanner noise and head-motion challenges during overt language tasks. We captured real-time information flows as coherent hemodynamic waves traveling over the cortical surface during listening, reading aloud, reciting, and oral cross-language interpreting tasks. We were able to observe the timing, location, direction, and surge of traveling waves in all language tasks, which were visualized as "brainstorms" on brain "weather" maps. The paths of hemodynamic traveling waves provide direct evidence for dual-stream models of the visual and auditory systems as well as logistics models for crossmodal and cross-language processing. Specifically, we have tracked down the step-by-step processing of written or spoken sentences first being received and processed by the visual or auditory streams, carried across language and domain-general cognitive regions, and finally delivered as overt speeches monitored through the auditory cortex, which gives a complete picture of information flows across the brain during natural language functioning. PRACTITIONER POINTS: Phase-encoded fMRI enables simultaneous imaging of high spatial and temporal resolution, capturing continuous spatiotemporal dynamics of the entire brain during real-time overt natural language tasks. Spatiotemporal traveling wave patterns provide direct evidence for constructing comprehensive and explicit models of human information processing. This study unlocks the potential of applying rapid phase-encoded fMRI to indirectly track the underlying neural information flows of sequential sensory, motor, and high-order cognitive processes.

Phase-encoded fMRI tracks down brainstorms of natural language processing with sub-second precision

The human language system interacts with cognitive and sensorimotor regions during natural language processing. However, where, when, and how these processes occur remain unclear. Existing noninvasive subtraction-based neuroimaging techniques cannot simultaneously achieve the spatial and temporal resolutions required to visualize ongoing information flows across the whole brain. Here we have developed phase-encoded designs to fully exploit the temporal information latent in functional magnetic resonance imaging (fMRI) data, as well as overcoming scanner noise and head-motion challenges during overt language tasks. We captured neural information flows as coherent waves traveling over the cortical surface during listening, reciting, and oral cross-language interpreting. The timing, location, direction, and surge of traveling waves, visualized as 'brainstorms' on brain 'weather' maps, reveal the functional and effective connectivity of the brain in action. These maps uncover the functional neuroanatomy of language perception and production and motivate the construction of finer-grained models of human information processing.

Topological Maps and Brain Computations From Low to High

We first briefly summarize data from microelectrode studies on visual maps in non-human primates and other mammals, and characterize differences among the features of the approximately topological maps in the three main sensory modalities. We then explore the almost 50% of human neocortex that contains straightforward topological visual, auditory, and somatomotor maps by presenting a new parcellation as well as a movie atlas of cortical area maps on the FreeSurfer average surface, fsaverage. Third, we review data on moveable map phenomena as well as a recent study showing that cortical activity during sensorimotor actions may involve spatially locally coherent traveling wave and bump activity. Finally, by analogy with remapping phenomena and sensorimotor activity, we speculate briefly on the testable possibility that coherent localized spatial activity patterns might be able to ‘escape’ from topologically mapped cortex during ‘serial assembly of content’ operations such as scene and language comprehension, to form composite ‘molecular’ patterns that can move across some cortical areas and possibly return to topologically mapped cortex to generate motor output there.

Attention module improves both performance and interpretability of four-dimensional functional magnetic resonance imaging decoding neural network

Decoding brain cognitive states from neuroimaging signals is an important topic in neuroscience. In recent years, deep neural networks (DNNs) have been recruited for multiple brain state decoding and achieved good performance. However, the open question of how to interpret the DNN black box remains unanswered. Capitalizing on advances in machine learning, we integrated attention modules into brain decoders to facilitate an in‐depth interpretation of DNN channels. A four‐dimensional (4D) convolution operation was also included to extract temporo‐spatial interaction within the fMRI signal. The experiments showed that the proposed model obtains a very high accuracy (97.4%) and outperforms previous researches on the seven different task benchmarks from the Human Connectome Project (HCP) dataset. The visualization analysis further illustrated the hierarchical emergence of task‐specific masks with depth. Finally, the model was retrained to regress individual traits within the HCP and to classify viewing images from the BOLD5000 dataset, respectively. Transfer learning also achieves good performance. Further visualization analysis shows that, after transfer learning, low‐level attention masks remained similar to the source domain, whereas high‐level attention masks changed adaptively. In conclusion, the proposed 4D model with attention module performed well and facilitated interpretation of DNNs, which is helpful for subsequent research.

The macaque ventral intraparietal area has expanded into three homologue human parietal areas

The macaque ventral intraparietal area (VIP) in the fundus of the intraparietal sulcus has been implicated in a diverse range of sensorimotor and cognitive functions such as motion processing, multisensory integration, processing of head peripersonal space, defensive behavior, and numerosity coding. Here, we exhaustively review macaque VIP function, cytoarchitectonics, and anatomical connectivity and integrate it with human studies that have attempted to identify a potential human VIP homologue. We show that human VIP research has consistently identified three, rather than one, bilateral parietal areas that each appear to subsume some, but not all, of the macaque area's functionality. Available evidence suggests that this human "VIP complex" has evolved as an expansion of the macaque area, but that some precursory specialization within macaque VIP has been previously overlooked. The three human areas are dominated, roughly, by coding the head or self in the environment, visual heading direction, and the peripersonal environment around the head, respectively. A unifying functional principle may be best described as prediction in space and time, linking VIP to state estimation as a key parietal sensorimotor function. VIP's expansive differentiation of head and self-related processing may have been key in the emergence of human bodily self-consciousness.

Coordination of multiple joints increases bilateral connectivity with ipsilateral sensorimotor cortices

Although most activities of daily life require simultaneous coordination of both proximal and distal joints, motor preparation during such movements has not been well studied. Previous results for motor preparation have focused on hand/finger movements. For simple hand/finger movements, results have found that such movements typically evoke activity primarily in the contralateral motor cortices. However, increasing the complexity of the finger movements, such as during a distal sequential finger-pressing task, leads to additional recruitment of ipsilateral resources. It has been suggested that this involvement of the ipsilateral hemisphere is critical for temporal coordination of distal joints. The goal of the current study was to examine whether increasing simultaneous coordination of multiple joints (both proximal and distal) leads to a similar increase in coupling with ipsilateral sensorimotor cortices during motor preparation compared to a simple distal movement such as hand opening. To test this possibility, 12 healthy individuals participated in a high-density EEG experiment in which they performed either hand opening or simultaneous hand opening while lifting at the shoulder on a robotic device. We quantified within- and cross-frequency cortical coupling across the sensorimotor cortex for the two tasks using dynamic causal modeling. Both hand opening and simultaneous hand opening while lifting at the shoulder elicited coupling from secondary motor areas to primary motor cortex within the contralateral hemisphere exclusively in the beta band, as well as from ipsilateral primary motor cortex. However, increasing the task complexity by combining hand opening while lifting at the shoulder also led to an increase in cross-frequency coupling within the ipsilateral hemisphere including theta, beta, and gamma frequencies, as well as a change in the coupling frequency of the interhemispheric coupling between the primary motor and premotor cortices. These findings demonstrate that increasing the demand of joint coordination between proximal and distal joints leads to increases in communication with the ipsilateral hemisphere as previously observed in distal sequential finger tasks.