Parietal maps of visual signals for bodily action planning

Two different mirror neuron pathways for social and non-social actions? A meta-analysis of fMRI studies

Mirror neurons (MNs) represent a class of neurons that are activated when performing or observing the same action. Given their role in social cognition and previous research in patients with psychiatric disorders, we proposed that the human MN system (MNS) might display different pathways for social and non-social actions. To examine this hypothesis, we conducted a comprehensive meta-analysis of 174 published human functional magnetic resonance imaging studies. Our findings confirmed the proposed hypothesis. Our results demonstrated that the non-social MN pathway exhibited a more classical pattern of frontoparietal activation, whereas the social MN pathway was activated less in the parietal lobe but more in the frontal lobe, limbic lobe, and sublobar regions. Additionally, our findings revealed a modulatory role of the effector (i.e. face and hands) within this framework: some areas exhibited effector-independent activation, while others did not. This novel subdivision provides valuable theoretical support for further investigations into the neural mechanisms underlying the MNS and its related disorders.

A model of task-level human stepping regulation yields semistable walking

A simple lateral dynamic walker, with swing leg dynamics and three adjustable input parameters, is used to study how motor regulation affects frontal-plane stepping. Motivated by experimental observations and phenomenological models, we imposed task-level multi-objective regulation targeting the walker's optimal lateral foot placement at each step. The regulator prioritizes achieving step width and lateral body position goals to varying degrees by choosing a mixture parameter. Our model thus integrates a lateral mechanical template, which captures the fundamental mechanics of frontal-plane walking, with a lateral motor regulation template, an empirically verified model of how humans manipulate lateral foot placements in a goal-directed manner. The model captures experimentally observed stepping fluctuation statistics and demonstrates how linear empirical models of stepping dynamics can emerge from first-principles nonlinear mechanics. We find that task-level regulation gives rise to a goal-equivalent manifold in the system's extended state space of mechanical states and inputs, a subset of which contains a continuum of period-1 gaits forming a semistable set: perturbations off of any of its gaits result in transients that return to the set, though typically to different gaits.

Modulation of multisensory nociceptive neurons in monkey cortical area 7b and behavioral correlates

Wide-range thermoreceptive neurons (WRT-EN) in monkey cortical area 7b that encoded innocuous and nocuous cutaneous thermal and threatening visuosensory stimulation with high fidelity were studied to identify their multisensory integrative response properties. Emphasis was given to characterizing the spatial and temporal effects of threatening visuosensory input on the thermal stimulus-response properties of these multisensory nociceptive neurons. Threatening visuosensory stimulation was most efficacious in modulating thermal evoked responses when presented as a downward ("looming"), spatially congruent, approaching and closely proximal target in relation to the somatosensory receptive field. Both temporal alignment and misalignment of spatially aligned threatening visual and thermal stimulation significantly increased mean discharge frequencies above those evoked by thermal stimulation alone, particularly at near noxious (43°C) and mildly noxious (45°C) temperatures. The enhanced multisensory discharge frequencies were equivalent to the discharge frequency evoked by overtly noxious thermal stimulation alone at 47°C (monkey pain tolerance threshold). A significant increase in behavioral mean escape frequency with shorter escape latency was evoked by multisensory stimulation at near noxious temperature (43°C), which was equivalent to that evoked by noxious stimulation alone (47°C). The remarkable concordance of elevating both neural discharge and escape frequency from a nonnociceptive and prepain level by near noxious thermal stimulation to a nociceptive and pain level by multisensory visual and near noxious thermal stimulation and integration is an elegantly designed defensive neural mechanism that in effect lowers both nociceptive response and pain thresholds to preemptively engage nocifensive behavior and, consequently, avert impending and actual injurious noxious thermal stimulation.NEW & NOTEWORTHY Multisensory nociceptive neurons in cortical area 7b are engaged in integration of threatening visuosensory and a wide range of innocuous and nocuous somatosensory (thermoreceptive) inputs. The enhancement of neuronal activity and escape behavior in monkey by multisensory integration is consistent and supportive of human psychophysical studies. The spatial features of visuosensory stimulation in peripersonal space in relation to somatic stimulation in personal space are critical to multisensory integration, nociception, nocifensive behavior, and pain.

A model of task-level human stepping regulation yields semistable walking

A simple lateral dynamic walker, with swing leg dynamics and three adjustable input parameters, is used to study how motor regulation affects frontal plane stepping. Motivated by experimental observations and phenomenological models, we imposed task-level multiobjective regulation targeting the walker's optimal lateral foot placement at each step. The regulator prioritizes achieving step width and lateral body position goals to varying degrees by choosing a mixture parameter. Our model thus integrates a lateral mechanical template, which captures fundamental mechanics of frontal-plane walking, with a lateral motor regulation template, an empirically verified model of how humans manipulate lateral foot placements in a goal-directed manner. The model captures experimentally observed stepping fluctuation statistics and demonstrates how linear empirical models of stepping dynamics can emerge from first-principles nonlinear mechanics. We find that task-level regulation gives rise to a goal equivalent manifold in the system's extended state space of mechanical states and inputs, a subset of which contains a continuum of period-1 gaits forming a semistable set: perturbations off of any of its gaits result in transients that return to the set, though typically to different gaits.

Selective activations and functional connectivities to the sight of faces, scenes, body parts and tools in visual and non-visual cortical regions leading to the human hippocampus

Connectivity maps are now available for the 360 cortical regions in the Human Connectome Project Multimodal Parcellation atlas. Here we add function to these maps by measuring selective fMRI activations and functional connectivity increases to stationary visual stimuli of faces, scenes, body parts and tools from 956 HCP participants. Faces activate regions in the ventrolateral visual cortical stream (FFC), in the superior temporal sulcus (STS) visual stream for face and head motion; and inferior parietal visual (PGi) and somatosensory (PF) regions. Scenes activate ventromedial visual stream VMV and PHA regions in the parahippocampal scene area; medial (7m) and lateral parietal (PGp) regions; and the reward-related medial orbitofrontal cortex. Body parts activate the inferior temporal cortex object regions (TE1p, TE2p); but also visual motion regions (MT, MST, FST); and the inferior parietal visual (PGi, PGs) and somatosensory (PF) regions; and the unpleasant-related lateral orbitofrontal cortex. Tools activate an intermediate ventral stream area (VMV3, VVC, PHA3); visual motion regions (FST); somatosensory (1, 2); and auditory (A4, A5) cortical regions. The findings add function to cortical connectivity maps; and show how stationary visual stimuli activate other cortical regions related to their associations, including visual motion, somatosensory, auditory, semantic, and orbitofrontal cortex value-related, regions.

Influence of gaze, vision, and memory on hand kinematics in a placement task

People usually reach for objects to place them in some position and orientation, but the placement component of this sequence is often ignored. For example, reaches are influenced by gaze position, visual feedback, and memory delays, but their influence on object placement is unclear. Here, we tested these factors in a task where participants placed and oriented a trapezoidal block against two-dimensional (2-D) visual templates displayed on a frontally located computer screen. In experiment 1, participants matched the block to three possible orientations: 0° (horizontal), +45° and -45°, with gaze fixated 10° to the left/right. The hand and template either remained illuminated (closed-loop), or visual feedback was removed (open-loop). Here, hand location consistently overshot the template relative to gaze, especially in the open-loop task; likewise, orientation was influenced by gaze position (depending on template orientation and visual feedback). In experiment 2, a memory delay was added, and participants sometimes performed saccades (toward, away from, or across the template). In this task, the influence of gaze on orientation vanished, but location errors were influenced by both template orientation and final gaze position. Contrary to our expectations, the previous saccade metrics also impacted placement overshoot. Overall, hand orientation was influenced by template orientation in a nonlinear fashion. These results demonstrate interactions between gaze and orientation signals in the planning and execution of hand placement and suggest different neural mechanisms for closed-loop, open-loop, and memory delay placement.NEW & NOTEWORTHY Eye-hand coordination studies usually focus on object acquisition, but placement is equally important. We investigated how gaze position influences object placement toward a 2-D template with different levels of visual feedback. Like reach, placement overestimated goal location relative to gaze and was influenced by previous saccade metrics. Gaze also modulated hand orientation, depending on template orientation and level of visual feedback. Gaze influence was feedback-dependent, with location errors having no significant effect after a memory delay.

Motor Intentions Decoded from fMRI Signals

Motor intention is a high-level brain function related to planning for movement. Although studies have shown that motor intentions can be decoded from brain signals before movement execution, it is unclear whether intentions relating to mental imagery of movement can be decoded. Here, we investigated whether differences in spatial and temporal patterns of brain activation were elicited by intentions to perform different types of motor imagery and whether the patterns could be used by a multivariate pattern classifier to detect such differential intentions. The results showed that it is possible to decode intentions before the onset of different types of motor imagery from functional MR signals obtained from fronto-parietal brain regions, such as the premotor cortex and posterior parietal cortex, while controlling for eye movements and for muscular activity of the hands. These results highlight the critical role played by the aforementioned brain regions in covert motor intentions. Moreover, they have substantial implications for rehabilitating patients with motor disabilities.

Mixed selectivity in monkey anterior intraparietal area during visual and motor processes

Classical studies suggest that the anterior intraparietal area (AIP) contributes to the encoding of specific information such as objects and actions of self and others, through a variety of neuronal classes, such as canonical, motor and mirror neurons. However, these studies typically focused on a single variable, leaving it unclear whether distinct sets of AIP neurons encode a single or multiple sources of information and how multimodal coding emerges. Here, we chronically recorded monkey AIP neurons in a variety of tasks and conditions classically employed in separate experiments. Most cells exhibited mixed selectivity for observed objects, executed actions, and observed actions, enhanced when this information came from the monkey's peripersonal working space. In contrast with the classical view, our findings indicate that multimodal coding emerges in AIP from partially-mixed selectivity of individual neurons for a variety of information relevant for planning actions directed to both physical objects and other subjects.

Two what, two where, visual cortical streams in humans

ROLLS, E. T. Two What, Two Where, Visual Cortical Streams in Humans. NEUROSCI BIOBEHAV REV 2024. Recent cortical connectivity investigations lead to new concepts about 'What' and 'Where' visual cortical streams in humans, and how they connect to other cortical systems. A ventrolateral 'What' visual stream leads to the inferior temporal visual cortex for object and face identity, and provides 'What' information to the hippocampal episodic memory system, the anterior temporal lobe semantic system, and the orbitofrontal cortex emotion system. A superior temporal sulcus (STS) 'What' visual stream utilising connectivity from the temporal and parietal visual cortex responds to moving objects and faces, and face expression, and connects to the orbitofrontal cortex for emotion and social behaviour. A ventromedial 'Where' visual stream builds feature combinations for scenes, and provides 'Where' inputs via the parahippocampal scene area to the hippocampal episodic memory system that are also useful for landmark-based navigation. The dorsal 'Where' visual pathway to the parietal cortex provides for actions in space, but also provides coordinate transforms to provide inputs to the parahippocampal scene area for self-motion update of locations in scenes in the dark or when the view is obscured.

The dorsal stream of visual processing and action-specific domains in parietal and frontal cortex in primates

This review summarizes our findings obtained from over 15 years of research on parietal-frontal networks involved in the dorsal stream of cortical processing. We have presented considerable evidence for the existence of similar, partially independent, parietal-frontal networks involved in specific motor actions in a number of primates. These networks are formed by connections between action-specific domains representing the same complex movement evoked by electrical microstimulation. Functionally matched domains in the posterior parietal (PPC) and frontal (M1-PMC) motor regions are hierarchically related. M1 seems to be a critical link in these networks, since the outputs of M1 are essential to the evoked behavior, whereas PPC and PMC mediate complex movements mostly via their connections with M1. Thus, lesioning or deactivating M1 domains selectively blocks matching PMC and PPC domains, while having limited impact on other domains. When pairs of domains are stimulated together, domains within the same parietal-frontal network (matching domains) are cooperative in evoking movements, while they are mainly competitive with other domains (mismatched domains) within the same set of cortical areas. We propose that the interaction of different functional domains in each cortical region (as well as in striatum) occurs mainly via mutual suppression. Thus, the domains at each level are in competition with each other for mediating one of several possible behavioral outcomes.

Hippocampal spatial view cells for memory and navigation, and their underlying connectivity in humans

Hippocampal and parahippocampal gyrus spatial view neurons in primates respond to the spatial location being looked at. The representation is allocentric, in that the responses are to locations "out there" in the world, and are relatively invariant with respect to retinal position, eye position, head direction, and the place where the individual is located. The underlying connectivity in humans is from ventromedial visual cortical regions to the parahippocampal scene area, leading to the theory that spatial view cells are formed by combinations of overlapping feature inputs self-organized based on their closeness in space. Thus, although spatial view cells represent "where" for episodic memory and navigation, they are formed by ventral visual stream feature inputs in the parahippocampal gyrus in what is the parahippocampal scene area. A second "where" driver of spatial view cells are parietal inputs, which it is proposed provide the idiothetic update for spatial view cells, used for memory recall and navigation when the spatial view details are obscured. Inferior temporal object "what" inputs and orbitofrontal cortex reward inputs connect to the human hippocampal system, and in macaques can be associated in the hippocampus with spatial view cell "where" representations to implement episodic memory. Hippocampal spatial view cells also provide a basis for navigation to a series of viewed landmarks, with the orbitofrontal cortex reward inputs to the hippocampus providing the goals for navigation, which can then be implemented by hippocampal connectivity in humans to parietal cortex regions involved in visuomotor actions in space. The presence of foveate vision and the highly developed temporal lobe for object and scene processing in primates including humans provide a basis for hippocampal spatial view cells to be key to understanding episodic memory in the primate and human hippocampus, and the roles of this system in primate including human navigation.

The human posterior parietal cortex: effective connectome, and its relation to function

The effective connectivity between 21 regions in the human posterior parietal cortex, and 360 cortical regions was measured in 171 Human Connectome Project (HCP) participants using the HCP atlas, and complemented with functional connectivity and diffusion tractography. Intraparietal areas LIP, VIP, MIP, and AIP have connectivity from early cortical visual regions, and to visuomotor regions such as the frontal eye fields, consistent with functions in eye saccades and tracking. Five superior parietal area 7 regions receive from similar areas and from the intraparietal areas, but also receive somatosensory inputs and connect with premotor areas including area 6, consistent with functions in performing actions to reach for, grasp, and manipulate objects. In the anterior inferior parietal cortex, PFop, PFt, and PFcm are mainly somatosensory, and PF in addition receives visuo-motor and visual object information, and is implicated in multimodal shape and body image representations. In the posterior inferior parietal cortex, PFm and PGs combine visuo-motor, visual object, and reward input and connect with the hippocampal system. PGi in addition provides a route to motion-related superior temporal sulcus regions involved in social interactions. PGp has connectivity with intraparietal regions involved in coordinate transforms and may be involved in idiothetic update of hippocampal visual scene representations.

The human posterior cingulate, retrosplenial, and medial parietal cortex effective connectome, and implications for memory and navigation

The human posterior cingulate, retrosplenial, and medial parietal cortex are involved in memory and navigation. The functional anatomy underlying these cognitive functions was investigated by measuring the effective connectivity of these Posterior Cingulate Division (PCD) regions in the Human Connectome Project-MMP1 atlas in 171 HCP participants, and complemented with functional connectivity and diffusion tractography. First, the postero-ventral parts of the PCD (31pd, 31pv, 7m, d23ab, and v23ab) have effective connectivity with the temporal pole, inferior temporal visual cortex, cortex in the superior temporal sulcus implicated in auditory and semantic processing, with the reward-related vmPFC and pregenual anterior cingulate cortex, with the inferior parietal cortex, and with the hippocampal system. This connectivity implicates it in hippocampal episodic memory, providing routes for "what," reward and semantic schema-related information to access the hippocampus. Second, the antero-dorsal parts of the PCD (especially 31a and 23d, PCV, and also RSC) have connectivity with early visual cortical areas including those that represent spatial scenes, with the superior parietal cortex, with the pregenual anterior cingulate cortex, and with the hippocampal system. This connectivity implicates it in the "where" component for hippocampal episodic memory and for spatial navigation. The dorsal-transitional-visual (DVT) and ProStriate regions where the retrosplenial scene area is located have connectivity from early visual cortical areas to the parahippocampal scene area, providing a ventromedial route for spatial scene information to reach the hippocampus. These connectivities provide important routes for "what," reward, and "where" scene-related information for human hippocampal episodic memory and navigation. The midcingulate cortex provides a route from the anterior dorsal parts of the PCD and the supracallosal part of the anterior cingulate cortex to premotor regions.

Gait control by the frontal lobe

The frontal lobe is crucial and contributes to controlling truncal motion, postural responses, and maintaining equilibrium and locomotion. The rich repertoire of frontal gait disorders gives some indication of this complexity. For human walking, it is necessary to simultaneously achieve at least two tasks, such as maintaining a bipedal upright posture and locomotion. Particularly, postural control plays an extremely significant role in enabling the subject to maintain stable gait behaviors to adapt to the environment. To achieve these requirements, the frontal cortex (1) uses cognitive information from the parietal, temporal, and occipital cortices, (2) creates plans and programs of gait behaviors, and (3) acts on the brainstem and spinal cord, where the core posture-gait mechanisms exist. Moreover, the frontal cortex enables one to achieve a variety of gait patterns in response to environmental changes by switching gait patterns from automatic routine to intentionally controlled and learning the new paradigms of gait strategy via networks with the basal ganglia, cerebellum, and limbic structures. This chapter discusses the role of each area of the frontal cortex in behavioral control and attempts to explain how frontal lobe controls walking with special reference to postural control.

A Large Video Set of Natural Human Actions for Visual and Cognitive Neuroscience Studies and Its Validation with fMRI

The investigation of the perception of others' actions and underlying neural mechanisms has been hampered by the lack of a comprehensive stimulus set covering the human behavioral repertoire. To fill this void, we present a video set showing 100 human actions recorded in natural settings, covering the human repertoire except for emotion-driven (e.g., sexual) actions and those involving implements (e.g., tools). We validated the set using fMRI and showed that observation of the 100 actions activated the well-established action observation network. We also quantified the videos' low-level visual features (luminance, optic flow, and edges). Thus, this comprehensive video set is a valuable resource for perceptual and neuronal studies.

Prefrontal and somatosensory-motor cortex effective connectivity in humans

Effective connectivity, functional connectivity, and tractography were measured between 57 cortical frontal and somatosensory regions and the 360 cortical regions in the Human Connectome Project (HCP) multimodal parcellation atlas for 171 HCP participants. A ventral somatosensory stream connects from 3b and 3a via 1 and 2 and then via opercular and frontal opercular regions to the insula, which then connects to inferior parietal PF regions. This stream is implicated in "what"-related somatosensory processing of objects and of the body and in combining with visual inputs in PF. A dorsal "action" somatosensory stream connects from 3b and 3a via 1 and 2 to parietal area 5 and then 7. Inferior prefrontal regions have connectivity with the inferior temporal visual cortex and orbitofrontal cortex, are implicated in working memory for "what" processing streams, and provide connectivity to language systems, including 44, 45, 47l, TPOJ1, and superior temporal visual area. The dorsolateral prefrontal cortex regions that include area 46 have connectivity with parietal area 7 and somatosensory inferior parietal regions and are implicated in working memory for actions and planning. The dorsal prefrontal regions, including 8Ad and 8Av, have connectivity with visual regions of the inferior parietal cortex, including PGs and PGi, and are implicated in visual and auditory top-down attention.

HD-tDCS of primary and higher-order motor cortex affects action word processing

The contribution of action-perception systems of the brain to lexical semantics remains controversial. Here, we used high-definition transcranial direct current stimulation (HD-tDCS) in healthy adults to examine the role of primary (left hand motor area; HMA) and higher-order (left anterior inferior parietal lobe; aIPL) action areas in action-related word processing (action verbs and manipulable nouns) compared to non-action-related control words (non-action verbs and non-manipulable nouns). We investigated stimulation-related effects at three levels of semantic processing: subliminal, implicit, and explicit. Broadly, we found that stimulation of HMA and aIPL resulted in relative facilitation of action-related language processing compared to non-action. HMA stimulation facilitated action verb processing in subliminal and implicit task contexts, suggesting that HMA helps represent action verbs even in semantically shallow tasks. HMA stimulation also facilitated manipulable noun comprehension in an explicit semantic task, suggesting that HMA contributes to manipulable noun comprehension when semantic demands are high. aIPL stimulation facilitated both manipulable noun and action verb processing during an implicit task. We suggest that both HMA and aIPL play a functional role in action semantics. HMA plays a general role in the semantics of actions and manipulable objects, while aIPL is important only when visuo-motor coordination is required for the action.

Multiple cortical visual streams in humans

The effective connectivity between 55 visual cortical regions and 360 cortical regions was measured in 171 HCP participants using the HCP-MMP atlas, and complemented with functional connectivity and diffusion tractography. A Ventrolateral Visual "What" Stream for object and face recognition projects hierarchically to the inferior temporal visual cortex, which projects to the orbitofrontal cortex for reward value and emotion, and to the hippocampal memory system. A Ventromedial Visual "Where" Stream for scene representations connects to the parahippocampal gyrus and hippocampus. An Inferior STS (superior temporal sulcus) cortex Semantic Stream receives from the Ventrolateral Visual Stream, from visual inferior parietal PGi, and from the ventromedial-prefrontal reward system and connects to language systems. A Dorsal Visual Stream connects via V2 and V3A to MT+ Complex regions (including MT and MST), which connect to intraparietal regions (including LIP, VIP and MIP) involved in visual motion and actions in space. It performs coordinate transforms for idiothetic update of Ventromedial Stream scene representations. A Superior STS cortex Semantic Stream receives visual inputs from the Inferior STS Visual Stream, PGi, and STV, and auditory inputs from A5, is activated by face expression, motion and vocalization, and is important in social behaviour, and connects to language systems.

More Than the Face: Representations of Bodies in the Inferior Temporal Cortex

Visual representations of bodies, in addition to those of faces, contribute to the recognition of con- and heterospecifics, to action recognition, and to nonverbal communication. Despite its importance, the neural basis of the visual analysis of bodies has been less studied than that of faces. In this article, I review what is known about the neural processing of bodies, focusing on the macaque temporal visual cortex. Early single-unit recording work suggested that the temporal visual cortex contains representations of body parts and bodies, with the dorsal bank of the superior temporal sulcus representing bodily actions. Subsequent functional magnetic resonance imaging studies in both humans and monkeys showed several temporal cortical regions that are strongly activated by bodies. Single-unit recordings in the macaque body patches suggest that these represent mainly body shape features. More anterior patches show a greater viewpoint-tolerant selectivity for body features, which may reflect a processing principle shared with other object categories, including faces. Expected final online publication date for the Annual Review of Vision Science, Volume 8 is September 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Do graspable objects always leave a motor signature? A study on memory traces

Several studies have reported the existence of reciprocal interactions between the type of motor activity physically performed on objects and the conceptual knowledge that is retained of them. Whether covert motor activity plays a similar effect is less clear. Certainly, objects are strong triggers for actions, and motor components can make the associated concepts more memorable. However, addition of an action-related memory trace may not always be automatic and could rather depend on 'how' objects are encountered. To test this hypothesis, we compared memory for objects that passive observers experienced as verbal labels (the word describing them), visual images (color photographs) and actions (pantomimes of object use). We predicted that the more direct the involvement of action-related representations the more effective would be the addition of a motor code to the experience and the more accurate would be the recall. Results showed that memory for objects presented as words i.e., a format that might only indirectly prime the sensorimotor system, was generally less accurate compared to memory for objects presented as photographs or pantomimes, which are more likely to directly elicit motor simulation processes. In addition, free recall of objects experienced as pantomimes was more accurate when these items afforded actions performed towards one's body than actions directed away from the body. We propose that covert motor activity can contribute to objects' memory, but the beneficial addition of a motor code to the experience is not necessarily automatic. An advantage is more likely to emerge when the observer is induced to take a first-person stance during the encoding phase, as may happen for objects affording actions directed towards the body, which obviously carry more relevance for the actor.