1
|
Zbären GA, Kapur M, Meissner SN, Wenderoth N. Inferring occluded projectile motion changes connectivity within a visuo-fronto-parietal network. Brain Struct Funct 2024:10.1007/s00429-024-02815-2. [PMID: 38914897 DOI: 10.1007/s00429-024-02815-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2023] [Accepted: 06/03/2024] [Indexed: 06/26/2024]
Abstract
Anticipating the behaviour of moving objects in the physical environment is essential for a wide range of daily actions. This ability is thought to rely on mental simulations and has been shown to involve frontoparietal and early visual areas. Yet, the connectivity patterns between these regions during intuitive physical inference remain largely unknown. In this study, participants underwent fMRI while performing a task requiring them to infer the parabolic trajectory of an occluded ball falling under Newtonian physics, and a control task. Building on our previous research showing that when solving the physical inference task, early visual areas encode task-specific and perception-like information about the inferred trajectory, the present study aimed to (i) identify regions that are functionally coupled with early visual areas during the physical inference task, and (ii) investigate changes in effective connectivity within this network of regions. We found that early visual areas are functionally connected to a set of parietal and premotor regions when inferring occluded trajectories. Using dynamic causal modelling, we show that predicting occluded trajectories is associated with changes in effective connectivity within a parieto-premotor network, which may drive internally generated early visual activity in a top-down fashion. These findings offer new insights into the interaction between early visual and frontoparietal regions during physical inference, contributing to our understanding of the neural mechanisms underlying the ability to predict physical outcomes.
Collapse
Affiliation(s)
- Gabrielle Aude Zbären
- Neural Control of Movement Lab, Department of Health Science and technology, ETH Zurich, Zurich, Switzerland.
| | - Manu Kapur
- Professorship for Learning Sciences and Higher Education, ETH Zurich, Zurich, Switzerland
| | - Sarah Nadine Meissner
- Neural Control of Movement Lab, Department of Health Science and technology, ETH Zurich, Zurich, Switzerland
| | - Nicole Wenderoth
- Neural Control of Movement Lab, Department of Health Science and technology, ETH Zurich, Zurich, Switzerland
- Future Health Technologies, Singapore-ETH Centre, Campus for Research Excellence And Technological Enterprise (CREATE), Singapore, Singapore
| |
Collapse
|
2
|
Mahowald K, Ivanova AA, Blank IA, Kanwisher N, Tenenbaum JB, Fedorenko E. Dissociating language and thought in large language models. Trends Cogn Sci 2024; 28:517-540. [PMID: 38508911 DOI: 10.1016/j.tics.2024.01.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2023] [Revised: 01/31/2024] [Accepted: 01/31/2024] [Indexed: 03/22/2024]
Abstract
Large language models (LLMs) have come closest among all models to date to mastering human language, yet opinions about their linguistic and cognitive capabilities remain split. Here, we evaluate LLMs using a distinction between formal linguistic competence (knowledge of linguistic rules and patterns) and functional linguistic competence (understanding and using language in the world). We ground this distinction in human neuroscience, which has shown that formal and functional competence rely on different neural mechanisms. Although LLMs are surprisingly good at formal competence, their performance on functional competence tasks remains spotty and often requires specialized fine-tuning and/or coupling with external modules. We posit that models that use language in human-like ways would need to master both of these competence types, which, in turn, could require the emergence of separate mechanisms specialized for formal versus functional linguistic competence.
Collapse
|
3
|
Friedrich J, Fischer MH, Raab M. Invariant representations in abstract concept grounding - the physical world in grounded cognition. Psychon Bull Rev 2024:10.3758/s13423-024-02522-3. [PMID: 38806790 DOI: 10.3758/s13423-024-02522-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 04/26/2024] [Indexed: 05/30/2024]
Abstract
Grounded cognition states that mental representations of concepts consist of experiential aspects. For example, the concept "cup" consists of the sensorimotor experiences from interactions with cups. Typical modalities in which concepts are grounded are: The sensorimotor system (including interoception), emotion, action, language, and social aspects. Here, we argue that this list should be expanded to include physical invariants (unchanging features of physical motion; e.g., gravity, momentum, friction). Research on physical reasoning consistently demonstrates that physical invariants are represented as fundamentally as other grounding substrates, and therefore should qualify. We assess several theories of concept representation (simulation, conceptual metaphor, conceptual spaces, predictive processing) and their positions on physical invariants. We find that the classic grounded cognition theories, simulation and conceptual metaphor theory, have not considered physical invariants, while conceptual spaces and predictive processing have. We conclude that physical invariants should be included into grounded cognition theories, and that the core mechanisms of simulation and conceptual metaphor theory are well suited to do this. Furthermore, conceptual spaces and predictive processing are very promising and should also be integrated with grounded cognition in the future.
Collapse
Affiliation(s)
- Jannis Friedrich
- German Sport University Cologne, Germany, Am Sportpark Müngersdorf 6, 50933, Cologne, Germany.
| | - Martin H Fischer
- Psychology Department, University of Potsdam, Karl-Liebknecht-Strasse 24-25, House 14 D - 14476, Potsdam-Golm, Germany
| | - Markus Raab
- German Sport University Cologne, Germany, Am Sportpark Müngersdorf 6, 50933, Cologne, Germany
| |
Collapse
|
4
|
Huang T, Liu J. A stochastic world model on gravity for stability inference. eLife 2024; 12:RP88953. [PMID: 38712832 DOI: 10.7554/elife.88953] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/08/2024] Open
Abstract
The fact that objects without proper support will fall to the ground is not only a natural phenomenon, but also common sense in mind. Previous studies suggest that humans may infer objects' stability through a world model that performs mental simulations with a priori knowledge of gravity acting upon the objects. Here we measured participants' sensitivity to gravity to investigate how the world model works. We found that the world model on gravity was not a faithful replica of the physical laws, but instead encoded gravity's vertical direction as a Gaussian distribution. The world model with this stochastic feature fit nicely with participants' subjective sense of objects' stability and explained the illusion that taller objects are perceived as more likely to fall. Furthermore, a computational model with reinforcement learning revealed that the stochastic characteristic likely originated from experience-dependent comparisons between predictions formed by internal simulations and the realities observed in the external world, which illustrated the ecological advantage of stochastic representation in balancing accuracy and speed for efficient stability inference. The stochastic world model on gravity provides an example of how a priori knowledge of the physical world is implemented in mind that helps humans operate flexibly in open-ended environments.
Collapse
Affiliation(s)
- Taicheng Huang
- Department of Psychological and Cognitive Sciences & Tsinghua Laboratory of Brain and Intelligence, Tsinghua University, Beijing, China
| | - Jia Liu
- Department of Psychological and Cognitive Sciences & Tsinghua Laboratory of Brain and Intelligence, Tsinghua University, Beijing, China
| |
Collapse
|
5
|
Fischer J. Physical reasoning is the missing link between action goals and kinematics: A comment on "An active inference model of hierarchical action understanding, learning, and imitation" by Proietti et al. Phys Life Rev 2024; 48:198-200. [PMID: 38350304 DOI: 10.1016/j.plrev.2023.08.017] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2023] [Accepted: 08/24/2023] [Indexed: 02/15/2024]
Affiliation(s)
- Jason Fischer
- Department of Psychological and Brain Sciences, Johns Hopkins University, Baltimore, MD, USA.
| |
Collapse
|
6
|
Yildirim I, Siegel MH, Soltani AA, Ray Chaudhuri S, Tenenbaum JB. Perception of 3D shape integrates intuitive physics and analysis-by-synthesis. Nat Hum Behav 2024; 8:320-335. [PMID: 37996497 DOI: 10.1038/s41562-023-01759-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2023] [Accepted: 10/12/2023] [Indexed: 11/25/2023]
Abstract
Many surface cues support three-dimensional shape perception, but humans can sometimes still see shape when these features are missing-such as when an object is covered with a draped cloth. Here we propose a framework for three-dimensional shape perception that explains perception in both typical and atypical cases as analysis-by-synthesis, or inference in a generative model of image formation. The model integrates intuitive physics to explain how shape can be inferred from the deformations it causes to other objects, as in cloth draping. Behavioural and computational studies comparing this account with several alternatives show that it best matches human observers (total n = 174) in both accuracy and response times, and is the only model that correlates significantly with human performance on difficult discriminations. We suggest that bottom-up deep neural network models are not fully adequate accounts of human shape perception, and point to how machine vision systems might achieve more human-like robustness.
Collapse
Affiliation(s)
- Ilker Yildirim
- Department of Psychology, Yale University, New Haven, CT, USA.
- Department of Statistics & Data Science, Yale University, New Haven, CT, USA.
- Wu-Tsai Institute, Yale University, New Haven, CT, USA.
| | - Max H Siegel
- Department of Brain & Cognitive Sciences, MIT, Cambridge, MA, USA.
- The Center for Brains, Minds, and Machines, MIT, Cambridge, MA, USA.
| | - Amir A Soltani
- Department of Brain & Cognitive Sciences, MIT, Cambridge, MA, USA
- The Center for Brains, Minds, and Machines, MIT, Cambridge, MA, USA
| | | | - Joshua B Tenenbaum
- Department of Brain & Cognitive Sciences, MIT, Cambridge, MA, USA.
- The Center for Brains, Minds, and Machines, MIT, Cambridge, MA, USA.
| |
Collapse
|
7
|
Mitko A, Navarro-Cebrián A, Cormiea S, Fischer J. A dedicated mental resource for intuitive physics. iScience 2024; 27:108607. [PMID: 38222113 PMCID: PMC10784689 DOI: 10.1016/j.isci.2023.108607] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2022] [Revised: 06/15/2023] [Accepted: 11/29/2023] [Indexed: 01/16/2024] Open
Abstract
Countless decisions and actions in daily life draw on a mental model of the physical structure and dynamics of the world - from stepping carefully around a patch of slippery pavement to stacking delicate produce in a shopping basket. People can make fast and accurate inferences about how physical interactions will unfold, but it remains unclear whether we do so by applying a general set of physical principles across scenarios, or instead by reasoning about the physics of individual scenarios in an ad-hoc fashion. Here, we hypothesized that humans possess a dedicated and flexible mental resource for physical inference, and we tested for such a resource using a battery of fine-tuned tasks to capture individual differences in performance. Despite varying scene contents across tasks, we found that performance was highly correlated among them and well-explained by a unitary intuitive physics resource, distinct from other facets of cognition such as spatial reasoning and working memory.
Collapse
Affiliation(s)
- Alex Mitko
- Department of Psychological and Brain Sciences, Johns Hopkins University, Baltimore, MD, USA
| | - Ana Navarro-Cebrián
- Department of Psychological and Brain Sciences, Johns Hopkins University, Baltimore, MD, USA
- Department of Psychology, University of Maryland, College Park, MD, USA
| | - Sarah Cormiea
- Department of Psychological and Brain Sciences, Johns Hopkins University, Baltimore, MD, USA
- Department of Neurology, University of Pennsylvania, Philadelphia, PA, USA
| | - Jason Fischer
- Department of Psychological and Brain Sciences, Johns Hopkins University, Baltimore, MD, USA
| |
Collapse
|
8
|
Karakose-Akbiyik S, Sussman O, Wurm MF, Caramazza A. The Role of Agentive and Physical Forces in the Neural Representation of Motion Events. J Neurosci 2024; 44:e1363232023. [PMID: 38050107 PMCID: PMC10860628 DOI: 10.1523/jneurosci.1363-23.2023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2023] [Revised: 11/14/2023] [Accepted: 11/19/2023] [Indexed: 12/06/2023] Open
Abstract
How does the brain represent information about motion events in relation to agentive and physical forces? In this study, we investigated the neural activity patterns associated with observing animated actions of agents (e.g., an agent hitting a chair) in comparison to similar movements of inanimate objects that were either shaped solely by the physics of the scene (e.g., gravity causing an object to fall down a hill and hit a chair) or initiated by agents (e.g., a visible agent causing an object to hit a chair). Using an fMRI-based multivariate pattern analysis (MVPA), this design allowed testing where in the brain the neural activity patterns associated with motion events change as a function of, or are invariant to, agentive versus physical forces behind them. A total of 29 human participants (nine male) participated in the study. Cross-decoding revealed a shared neural representation of animate and inanimate motion events that is invariant to agentive or physical forces in regions spanning frontoparietal and posterior temporal cortices. In contrast, the right lateral occipitotemporal cortex showed a higher sensitivity to agentive events, while the left dorsal premotor cortex was more sensitive to information about inanimate object events that were solely shaped by the physics of the scene.
Collapse
Affiliation(s)
| | - Oliver Sussman
- Department of Psychology, Harvard University, Cambridge, Massachusetts 02138
| | - Moritz F Wurm
- Center for Mind/Brain Sciences - CIMeC, University of Trento, 38068 Rovereto, Italy
| | - Alfonso Caramazza
- Department of Psychology, Harvard University, Cambridge, Massachusetts 02138
- Center for Mind/Brain Sciences - CIMeC, University of Trento, 38068 Rovereto, Italy
| |
Collapse
|
9
|
Ilardi CR, La Marra M, Amato R, Di Cecca A, Di Maio G, Ciccarelli G, Migliaccio M, Cavaliere C, Federico G. The "Little Circles Test" (LCT): a dusted-off tool for assessing fine visuomotor function. Aging Clin Exp Res 2023; 35:2807-2820. [PMID: 37910290 DOI: 10.1007/s40520-023-02571-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/19/2023] [Accepted: 09/18/2023] [Indexed: 11/03/2023]
Abstract
BACKGROUND The fine visuomotor function is commonly impaired in several neurological conditions. However, there is a scarcity of reliable neuropsychological tools to assess such a critical domain. AIMS The aim of this study is to explore the psychometric properties and provide normative data for the Visual-Motor Speed and Precision Test (VMSPT). RESULTS Our normative sample included 220 participants (130 females) aged 18-86 years (mean education = 15.24 years, SD = 3.98). Results showed that raw VMSPT scores were affected by higher age and lower education. No effect of sex or handedness was shown. Age- and education-based norms were provided. VMSPT exhibited weak-to-strong correlations with well-known neuropsychological tests, encompassing a wide range of cognitive domains of clinical relevance. By gradually intensifying the cognitive demands, the test becomes an indirect, performance-oriented measure of executive functioning. Finally, VMSPT seems proficient in capturing the speed-accuracy trade-off typically observed in the aging population. CONCLUSIONS This study proposes the initial standardization of a versatile, time-efficient, and cost-effective neuropsychological tool for assessing fine visuomotor coordination. We propose renaming the VMSPT as the more approachable "Little Circles Test" (LCT).
Collapse
Affiliation(s)
| | - Marco La Marra
- Department of Experimental Medicine, University of Campania "Luigi Vanvitelli", Naples, Italy
| | - Raffaella Amato
- Department of Experimental Medicine, University of Campania "Luigi Vanvitelli", Naples, Italy
| | - Angelica Di Cecca
- IRCCS SYNLAB SDN S.P.A., Via Emanuele Gianturco 113, 80143, Naples, Italy
| | - Girolamo Di Maio
- Department of Experimental Medicine, University of Campania "Luigi Vanvitelli", Naples, Italy
| | | | - Miriana Migliaccio
- IRCCS SYNLAB SDN S.P.A., Via Emanuele Gianturco 113, 80143, Naples, Italy
| | - Carlo Cavaliere
- IRCCS SYNLAB SDN S.P.A., Via Emanuele Gianturco 113, 80143, Naples, Italy
| | - Giovanni Federico
- IRCCS SYNLAB SDN S.P.A., Via Emanuele Gianturco 113, 80143, Naples, Italy
| |
Collapse
|
10
|
Nayebi A, Rajalingham R, Jazayeri M, Yang GR. Neural Foundations of Mental Simulation: Future Prediction of Latent Representations on Dynamic Scenes. ARXIV 2023:arXiv:2305.11772v2. [PMID: 37292459 PMCID: PMC10246064] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
Humans and animals have a rich and flexible understanding of the physical world, which enables them to infer the underlying dynamical trajectories of objects and events, plausible future states, and use that to plan and anticipate the consequences of actions. However, the neural mechanisms underlying these computations are unclear. We combine a goal-driven modeling approach with dense neurophysiological data and high-throughput human behavioral readouts that contain thousands of comparisons to directly impinge on this question. Specifically, we construct and evaluate several classes of sensory-cognitive networks to predict the future state of rich, ethologically-relevant environments, ranging from self-supervised end-to-end models with pixel-wise or object-slot objectives, to models that future predict in the latent space of purely static image-pretrained or dynamic video-pretrained foundation models. We find that "scale is not all you need", and that many state-of-the-art machine learning models fail to perform well on our neural and behavioral benchmarks for future prediction. In fact, only one class of models matches these data well overall. We find that neural responses are currently best predicted by models trained to predict the future state of their environment in the latent space of pretrained foundation models optimized for dynamic scenes in a self-supervised manner. These models also approach the neurons' ability to predict the environmental state variables that are visually hidden from view, despite not being explicitly trained to do so. Finally, we find that not all foundation model latents are equal. Notably, models that future predict in the latent space of video foundation models that are optimized to support a diverse range of egocentric sensorimotor tasks, reasonably match both human behavioral error patterns and neural dynamics across all environmental scenarios that we were able to test. Overall, these findings suggest that the neural mechanisms and behaviors of primate mental simulation have strong inductive biases associated with them, and are thus far most consistent with being optimized to future predict on reusable visual representations that are useful for Embodied AI more generally.
Collapse
Affiliation(s)
- Aran Nayebi
- McGovern Institute for Brain Research, MIT; Cambridge, MA 02139
| | - Rishi Rajalingham
- McGovern Institute for Brain Research, MIT; Cambridge, MA 02139
- Reality Labs, Meta; 390 9th Ave, New York, NY 10001
| | - Mehrdad Jazayeri
- McGovern Institute for Brain Research, MIT; Cambridge, MA 02139
- Department of Brain and Cognitive Sciences, MIT; Cambridge, MA 02139
| | - Guangyu Robert Yang
- McGovern Institute for Brain Research, MIT; Cambridge, MA 02139
- Department of Brain and Cognitive Sciences, MIT; Cambridge, MA 02139
- Department of Electrical Engineering and Computer Science, MIT; Cambridge, MA 02139
| |
Collapse
|
11
|
Su WC, Culotta M, Mueller J, Tsuzuki D, Bhat AN. Autism-Related Differences in Cortical Activation When Observing, Producing, and Imitating Communicative Gestures: An fNIRS Study. Brain Sci 2023; 13:1284. [PMID: 37759885 PMCID: PMC10527424 DOI: 10.3390/brainsci13091284] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2023] [Revised: 08/16/2023] [Accepted: 08/28/2023] [Indexed: 09/29/2023] Open
Abstract
Children with autism spectrum disorder (ASD) have difficulties in gestural communication during social interactions. However, the neural mechanisms involved in naturalistic gestural communication remain poorly understood. In this study, cortical activation patterns associated with gestural communication were examined in thirty-two children with and without ASD (mean age: 11.0 years, SE: 0.6 years). Functional near-infrared spectroscopy (fNIRS) was used to record cortical activation while children produced, observed, or imitated communicative gestures. Children with ASD demonstrated more spatial and temporal errors when performing and imitating communicative gestures. Although both typically developing (TD) children and children with ASD showed left-lateralized cortical activation during gesture production, children with ASD showed hyperactivation in the middle/inferior frontal gyrus (MIFG) during observation and imitation, and hypoactivation in the middle/superior temporal gyrus (MSTG) during gesture production compared to their TD peers. More importantly, children with ASD exhibited greater MSTG activation during imitation than during gesture production, suggesting that imitation could be an effective intervention strategy to engage cortical regions crucial for processing and producing gestures. Our study provides valuable insights into the neural mechanisms underlying gestural communication difficulties in ASD, while also identifying potential neurobiomarkers that could serve as objective measures for evaluating intervention effectiveness in children with ASD.
Collapse
Affiliation(s)
- Wan-Chun Su
- Department of Physical Therapy, University of Delaware, Newark, DE 19713, USA; (W.-C.S.); (M.C.)
- Biomechanics and Movement Science Program, College of Health Sciences, University of Delaware, Newark, DE 19713, USA
| | - McKenzie Culotta
- Department of Physical Therapy, University of Delaware, Newark, DE 19713, USA; (W.-C.S.); (M.C.)
- Biomechanics and Movement Science Program, College of Health Sciences, University of Delaware, Newark, DE 19713, USA
| | - Jessica Mueller
- Department of Behavioral Health, Swank Autism Center, Nemours Children’s Hospital, Wilmington, DE 19803, USA;
| | - Daisuke Tsuzuki
- Department of Information Sciences, Kochi University, Kochi 780-8520, Japan;
| | - Anjana N. Bhat
- Department of Physical Therapy, University of Delaware, Newark, DE 19713, USA; (W.-C.S.); (M.C.)
- Biomechanics and Movement Science Program, College of Health Sciences, University of Delaware, Newark, DE 19713, USA
- Interdisciplinary Neuroscience Graduate Program, Department of Psychological and Brain Sciences, University of Delaware, Newark, DE 19713, USA
| |
Collapse
|
12
|
Emonds AMX, Srinath R, Nielsen KJ, Connor CE. Object representation in a gravitational reference frame. eLife 2023; 12:e81701. [PMID: 37561119 PMCID: PMC10414968 DOI: 10.7554/elife.81701] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2022] [Accepted: 07/19/2023] [Indexed: 08/11/2023] Open
Abstract
When your head tilts laterally, as in sports, reaching, and resting, your eyes counterrotate less than 20%, and thus eye images rotate, over a total range of about 180°. Yet, the world appears stable and vision remains normal. We discovered a neural strategy for rotational stability in anterior inferotemporal cortex (IT), the final stage of object vision in primates. We measured object orientation tuning of IT neurons in macaque monkeys tilted +25 and -25° laterally, producing ~40° difference in retinal image orientation. Among IT neurons with consistent object orientation tuning, 63% remained stable with respect to gravity across tilts. Gravitational tuning depended on vestibular/somatosensory but also visual cues, consistent with previous evidence that IT processes scene cues for gravity's orientation. In addition to stability across image rotations, an internal gravitational reference frame is important for physical understanding of a world where object position, posture, structure, shape, movement, and behavior interact critically with gravity.
Collapse
Affiliation(s)
- Alexandriya MX Emonds
- Department of Biomedical Engineering, Johns Hopkins University School of MedicineBaltimoreUnited States
- Zanvyl Krieger Mind/Brain Institute, Johns Hopkins UniversityBaltimoreUnited States
| | - Ramanujan Srinath
- Zanvyl Krieger Mind/Brain Institute, Johns Hopkins UniversityBaltimoreUnited States
- Solomon H. Snyder Department of Neuroscience, Johns Hopkins University School of MedicineBaltimoreUnited States
| | - Kristina J Nielsen
- Zanvyl Krieger Mind/Brain Institute, Johns Hopkins UniversityBaltimoreUnited States
- Solomon H. Snyder Department of Neuroscience, Johns Hopkins University School of MedicineBaltimoreUnited States
| | - Charles E Connor
- Zanvyl Krieger Mind/Brain Institute, Johns Hopkins UniversityBaltimoreUnited States
- Solomon H. Snyder Department of Neuroscience, Johns Hopkins University School of MedicineBaltimoreUnited States
| |
Collapse
|
13
|
Gweon H, Fan J, Kim B. Socially intelligent machines that learn from humans and help humans learn. PHILOSOPHICAL TRANSACTIONS. SERIES A, MATHEMATICAL, PHYSICAL, AND ENGINEERING SCIENCES 2023; 381:20220048. [PMID: 37271177 DOI: 10.1098/rsta.2022.0048] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/19/2022] [Accepted: 04/17/2023] [Indexed: 06/06/2023]
Abstract
A hallmark of human intelligence is the ability to understand and influence other minds. Humans engage in inferential social learning (ISL) by using commonsense psychology to learn from others and help others learn. Recent advances in artificial intelligence (AI) are raising new questions about the feasibility of human-machine interactions that support such powerful modes of social learning. Here, we envision what it means to develop socially intelligent machines that can learn, teach, and communicate in ways that are characteristic of ISL. Rather than machines that simply predict human behaviours or recapitulate superficial aspects of human sociality (e.g. smiling, imitating), we should aim to build machines that can learn from human inputs and generate outputs for humans by proactively considering human values, intentions and beliefs. While such machines can inspire next-generation AI systems that learn more effectively from humans (as learners) and even help humans acquire new knowledge (as teachers), achieving these goals will also require scientific studies of its counterpart: how humans reason about machine minds and behaviours. We close by discussing the need for closer collaborations between the AI/ML and cognitive science communities to advance a science of both natural and artificial intelligence. This article is part of a discussion meeting issue 'Cognitive artificial intelligence'.
Collapse
Affiliation(s)
- Hyowon Gweon
- Department of Psychology, Stanford University, Stanford, CA 94305, USA
| | - Judith Fan
- Department of Psychology, Stanford University, Stanford, CA 94305, USA
- Department of Psychology, University of California, San Diego, CA 92093, USA
| | - Been Kim
- Google Research, Mountain View, CA 94043, USA
| |
Collapse
|
14
|
Karakose-Akbiyik S, Caramazza A, Wurm MF. A shared neural code for the physics of actions and object events. Nat Commun 2023; 14:3316. [PMID: 37286553 DOI: 10.1038/s41467-023-39062-8] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2022] [Accepted: 05/28/2023] [Indexed: 06/09/2023] Open
Abstract
Observing others' actions recruits frontoparietal and posterior temporal brain regions - also called the action observation network. It is typically assumed that these regions support recognizing actions of animate entities (e.g., person jumping over a box). However, objects can also participate in events with rich meaning and structure (e.g., ball bouncing over a box). So far, it has not been clarified which brain regions encode information specific to goal-directed actions or more general information that also defines object events. Here, we show a shared neural code for visually presented actions and object events throughout the action observation network. We argue that this neural representation captures the structure and physics of events regardless of animacy. We find that lateral occipitotemporal cortex encodes information about events that is also invariant to stimulus modality. Our results shed light onto the representational profiles of posterior temporal and frontoparietal cortices, and their roles in encoding event information.
Collapse
Affiliation(s)
| | - Alfonso Caramazza
- Department of Psychology, Harvard University, Cambridge, MA, USA
- Center for Mind/Brain Sciences - CIMeC, University of Trento, Rovereto, Italy
| | - Moritz F Wurm
- Center for Mind/Brain Sciences - CIMeC, University of Trento, Rovereto, Italy
| |
Collapse
|
15
|
Su WC, Culotta M, Mueller J, Tsuzuki D, Bhat A. fNIRS-Based Differences in Cortical Activation during Tool Use, Pantomimed Actions, and Meaningless Actions between Children with and without Autism Spectrum Disorder (ASD). Brain Sci 2023; 13:876. [PMID: 37371356 DOI: 10.3390/brainsci13060876] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/10/2023] [Revised: 05/16/2023] [Accepted: 05/24/2023] [Indexed: 06/29/2023] Open
Abstract
Children with autism spectrum disorder (ASD) have difficulties with tool use and pantomime actions. The current study utilized functional near-infrared spectroscopy (fNIRS) to examine the neural mechanisms underlying these gestural difficulties. Thirty-one children with and without ASD (age (mean ± SE) = 11.0 ± 0.6) completed a naturalistic peg-hammering task using an actual hammer (hammer condition), pantomiming hammering actions (pantomime condition), and performing meaningless actions with similar joint motions (meaningless condition). Children with ASD exhibited poor praxis performance (praxis error: TD = 17.9 ± 1.7; ASD = 27.0 ± 2.6, p < 0.01), which was significantly correlated with their cortical activation (R = 0.257 to 0.543). Both groups showed left-lateralized activation, but children with ASD demonstrated more bilateral activation during all gestural conditions. Compared to typically developing children, children with ASD showed hyperactivation of the inferior parietal lobe and hypoactivation of the middle/inferior frontal and middle/superior temporal regions. Our findings indicate intact technical reasoning (typical left-IPL activation) but atypical visuospatial and proprioceptive processing (hyperactivation of the right IPL) during tool use in children with ASD. These results have important implications for clinicians and researchers, who should focus on facilitating/reducing the burden of visuospatial and proprioceptive processing in children with ASD. Additionally, fNIRS-related biomarkers could be used for early identification through early object play/tool use and to examine neural effects following gesture-based interventions.
Collapse
Affiliation(s)
- Wan-Chun Su
- Department of Physical Therapy, University of Delaware, Newark, DE 19713, USA
- Biomechanics & Movement Science Program, College of Health Sciences, University of Delaware, Newark, DE 19713, USA
| | - McKenzie Culotta
- Department of Physical Therapy, University of Delaware, Newark, DE 19713, USA
- Biomechanics & Movement Science Program, College of Health Sciences, University of Delaware, Newark, DE 19713, USA
| | - Jessica Mueller
- Department of Behavioral Health, Swank Autism Center, A. I. du Pont Nemours Children's Hospital, Wilmington, DE 19803, USA
| | - Daisuke Tsuzuki
- Department of Information Science, Faculty of Science and Technology, Kochi University, Kochi 780-8520, Japan
| | - Anjana Bhat
- Department of Physical Therapy, University of Delaware, Newark, DE 19713, USA
- Biomechanics & Movement Science Program, College of Health Sciences, University of Delaware, Newark, DE 19713, USA
- Interdisciplinary Neuroscience Graduate (ING) Program, Department of Psychological & Brain Sciences, University of Delaware, Newark, DE 19716, USA
| |
Collapse
|
16
|
Boger T, Ullman T. What is "Where": Physical Reasoning Informs Object Location. Open Mind (Camb) 2023; 7:130-140. [PMID: 37416073 PMCID: PMC10320814 DOI: 10.1162/opmi_a_00075] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2022] [Accepted: 03/21/2023] [Indexed: 07/08/2023] Open
Abstract
A central puzzle the visual system tries to solve is: "what is where?" While a great deal of research attempts to model object recognition ("what"), a comparatively smaller body of work seeks to model object location ("where"), especially in perceiving everyday objects. How do people locate an object, right now, in front of them? In three experiments collecting over 35,000 judgements on stimuli spanning different levels of realism (line drawings, real images, and crude forms), participants clicked "where" an object is, as if pointing to it. We modeled their responses with eight different methods, including both human response-based models (judgements of physical reasoning, spatial memory, free-response "click anywhere" judgements, and judgements of where people would grab the object), and image-based models (uniform distributions over the image, convex hull, saliency map, and medial axis). Physical reasoning was the best predictor of "where," performing significantly better than even spatial memory and free-response judgements. Our results offer insight into the perception of object locations while also raising interesting questions about the relationship between physical reasoning and visual perception.
Collapse
Affiliation(s)
- Tal Boger
- Department of Psychology, Yale University, New Haven, CT, USA
| | - Tomer Ullman
- Department of Psychology, Harvard University, Cambridge, MA, USA
| |
Collapse
|
17
|
Marciniak Dg Agra K, Dg Agra P. F = ma. Is the macaque brain Newtonian? Cogn Neuropsychol 2023; 39:376-408. [PMID: 37045793 DOI: 10.1080/02643294.2023.2191843] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/14/2023]
Abstract
Intuitive Physics, the ability to anticipate how the physical events involving mass objects unfold in time and space, is a central component of intelligent systems. Intuitive physics is a promising tool for gaining insight into mechanisms that generalize across species because both humans and non-human primates are subject to the same physical constraints when engaging with the environment. Physical reasoning abilities are widely present within the animal kingdom, but monkeys, with acute 3D vision and a high level of dexterity, appreciate and manipulate the physical world in much the same way humans do.
Collapse
Affiliation(s)
- Karolina Marciniak Dg Agra
- The Rockefeller University, Laboratory of Neural Circuits, New York, NY, USA
- Center for Brain, Minds and Machines, Cambridge, MA, USA
| | - Pedro Dg Agra
- The Rockefeller University, Laboratory of Neural Circuits, New York, NY, USA
- Center for Brain, Minds and Machines, Cambridge, MA, USA
| |
Collapse
|
18
|
Osiurak F, Claidière N, Federico G. Cultural cognition and technology: Mechanical actions speak louder than bodily actions: Comment on "Blind alleys and fruitful pathways in the comparative study of cultural cognition" by Andrew Whiten. Phys Life Rev 2023; 44:141-144. [PMID: 36640588 DOI: 10.1016/j.plrev.2022.12.013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2022] [Accepted: 12/20/2022] [Indexed: 12/29/2022]
Affiliation(s)
- François Osiurak
- Laboratoire d'Étude des Mécanismes Cognitifs, Université de Lyon, 5 avenue Pierre Mendès France, 69676 Bron Cedex, France; Institut Universitaire de France, 1 rue Descartes, 75231 Paris Cedex 5, France.
| | - Nicolas Claidière
- Aix-Marseille Univ, CNRS, LPC, 3 Place Victor Hugo, 13331 Marseille, France
| | - Giovanni Federico
- IRCCS Synlab SDN S.p.A., Via Emanuele Gianturco 113, 80143, Naples, Italy; Laboratory of Experimental Psychology, Suor Orsola Benincasa University, Via Suor Orsola 10, 80135, Naples, Italy; Department of Psychology, University of Campania "Luigi Vanvitelli", Viale Ellittico 31, 81100, Caserta, Italy
| |
Collapse
|
19
|
Osiurak F, Claidière N, Federico G. Bringing cumulative technological culture beyond copying versus reasoning. Trends Cogn Sci 2023; 27:30-42. [PMID: 36283920 DOI: 10.1016/j.tics.2022.09.024] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2022] [Revised: 09/28/2022] [Accepted: 09/29/2022] [Indexed: 11/06/2022]
Abstract
The dominant view of cumulative technological culture suggests that high-fidelity transmission rests upon a high-fidelity copying ability, which allows individuals to reproduce the tool-use actions performed by others without needing to understand them (i.e., without causal reasoning). The opposition between copying versus reasoning is well accepted but with little supporting evidence. In this article, we investigate this distinction by examining the cognitive science literature on tool use. Evidence indicates that the ability to reproduce others' tool-use actions requires causal understanding, which questions the copying versus reasoning distinction and the cognitive reality of the so-called copying ability. We conclude that new insights might be gained by considering causal understanding as a key driver of cumulative technological culture.
Collapse
Affiliation(s)
- François Osiurak
- Laboratoire d'Étude des Mécanismes Cognitifs, Université de Lyon, 5 avenue Pierre Mendès France, 69676 Bron Cedex, France; Institut Universitaire de France, 1 rue Descartes, 75231 Paris Cedex 5, France.
| | - Nicolas Claidière
- Aix-Marseille Univ, CNRS, LPC, 3 Place Victor Hugo, 13331 Marseille, France
| | - Giovanni Federico
- IRCCS Synlab SDN S.p.A., Via Emanuele Gianturco 113, 80143, Naples, Italy
| |
Collapse
|
20
|
Rajalingham R, Piccato A, Jazayeri M. Recurrent neural networks with explicit representation of dynamic latent variables can mimic behavioral patterns in a physical inference task. Nat Commun 2022; 13:5865. [PMID: 36195614 PMCID: PMC9532407 DOI: 10.1038/s41467-022-33581-6] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2021] [Accepted: 09/22/2022] [Indexed: 11/09/2022] Open
Abstract
Primates can richly parse sensory inputs to infer latent information. This ability is hypothesized to rely on establishing mental models of the external world and running mental simulations of those models. However, evidence supporting this hypothesis is limited to behavioral models that do not emulate neural computations. Here, we test this hypothesis by directly comparing the behavior of primates (humans and monkeys) in a ball interception task to that of a large set of recurrent neural network (RNN) models with or without the capacity to dynamically track the underlying latent variables. Humans and monkeys exhibit similar behavioral patterns. This primate behavioral pattern is best captured by RNNs endowed with dynamic inference, consistent with the hypothesis that the primate brain uses dynamic inferences to support flexible physical predictions. Moreover, our work highlights a general strategy for using model neural systems to test computational hypotheses of higher brain function.
Collapse
Affiliation(s)
- Rishi Rajalingham
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Building 46, 43 Vassar St., Cambridge, MA, 02139, USA
| | - Aída Piccato
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Building 46, 43 Vassar St., Cambridge, MA, 02139, USA.,Department of Brain & Cognitive Sciences, Massachusetts Institute of Technology, Building 46, 43 Vassar St., Cambridge, MA, 02139-4307, USA
| | - Mehrdad Jazayeri
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Building 46, 43 Vassar St., Cambridge, MA, 02139, USA. .,Department of Brain & Cognitive Sciences, Massachusetts Institute of Technology, Building 46, 43 Vassar St., Cambridge, MA, 02139-4307, USA.
| |
Collapse
|
21
|
Choi JS, Choi MH. A study on brain neuronal activation based on the load in upper limb exercise (STROBE). Medicine (Baltimore) 2022; 101:e30761. [PMID: 36197190 PMCID: PMC9509160 DOI: 10.1097/md.0000000000030761] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/25/2022] Open
Abstract
This study aimed to determine the level of brain activation in separate regions, including the lobes, cerebellum, and limbic system, depending on the weight of an object during elbow flexion and extension exercise using functional magnetic resonance imaging (fMRI). The study was conducted on ten male undergraduates (22.4 ± 1.2 years). The functional images of the brain were obtained using the 3T MRI. The participants performed upper limb flexion and extension exercise at a constant speed and as the weight of the object for lifting was varied (0 g and 1000 g). The experiment consisted of four blocks that constituted 8 minutes. Each block was designed to comprise a rest phase (1 minute) and a lifting phase (1 minute). The results showed that, in the parietal lobe, the activation was higher for the 0 g-motion condition than for the 1000 g-motion condition; however, in the occipital lobe, cerebellum, sub-lobar, and limbic system, the activation was higher for the 1000 g-motion condition than for the 0 g-motion condition. The brain region for the perception of object weight was identified as the ventral area (occipital, temporal, and frontal lobe), and the activation of the ventral pathway is suggested to have increased as the object came into vision and as its shape, size, and weight were perceived. For holding an object in hand, compared to not holding it, the exercise load was greater for controlling the motion to maintain the posture (arm angle at 90°), controlling the speed to repeat the motion at a constant speed, and producing an accurate posing. Therefore, to maintain such varied conditions, the activation level increased in the regions associated with control and regulation through the motion coordination from vision to arm movements (control of muscles). A characteristic reduced activation was observed in the regions associated with visuo-vestibular interaction and voluntary movement when the exercise involved lifting a 1000-g object compared to the exercise without object lifting.
Collapse
Affiliation(s)
- Jin-Seung Choi
- Biomedical Engineering, Research Institute of Biomedical Engineering, School of ICT Convergence Engineering, College of Science and Technology, Konkuk University, Chungju, South Korea
| | - Mi-Hyun Choi
- Biomedical Engineering, Research Institute of Biomedical Engineering, School of ICT Convergence Engineering, College of Science and Technology, Konkuk University, Chungju, South Korea
- *Correspondence: Mi-Hyun Choi, Biomedical Engineering, Research Institute of Biomedical Engineering, School of ICT Convergence Engineering, College of Science and Technology, Konkuk University, 268 Chungwon-daero, Chungju-si, Chungcheongbuk-do, 27478, South Korea (e-mail: )
| |
Collapse
|
22
|
Federico G, Reynaud E, Navarro J, Lesourd M, Gaujoux V, Lamberton F, Ibarrola D, Cavaliere C, Alfano V, Aiello M, Salvatore M, Seguin P, Schnebelen D, Brandimonte MA, Rossetti Y, Osiurak F. The cortical thickness of the area PF of the left inferior parietal cortex mediates technical-reasoning skills. Sci Rep 2022; 12:11840. [PMID: 35821259 PMCID: PMC9276675 DOI: 10.1038/s41598-022-15587-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2021] [Accepted: 06/27/2022] [Indexed: 11/23/2022] Open
Abstract
Most recent research highlights how a specific form of causal understanding, namely technical reasoning, may support the increasing complexity of tools and techniques developed by humans over generations, i.e., the cumulative technological culture (CTC). Thus, investigating the neurocognitive foundations of technical reasoning is essential to comprehend the emergence of CTC in our lineage. Whereas functional neuroimaging evidence started to highlight the critical role of the area PF of the left inferior parietal cortex (IPC) in technical reasoning, no studies explored the links between the structural characteristics of such a brain region and technical reasoning skills. Therefore, in this study, we assessed participants’ technical-reasoning performance by using two ad-hoc psycho-technical tests; then, we extracted from participants’ 3 T T1-weighted magnetic-resonance brain images the cortical thickness (i.e., a volume-related measure which is associated with cognitive performance as reflecting the size, density, and arrangement of cells in a brain region) of all the IPC regions for both hemispheres. We found that the cortical thickness of the left area PF predicts participants’ technical-reasoning performance. Crucially, we reported no correlations between technical reasoning and the other IPC regions, possibly suggesting the specificity of the left area PF in generating technical knowledge. We discuss these findings from an evolutionary perspective, by speculating about how the evolution of parietal lobes may have supported the emergence of technical reasoning in our lineage.
Collapse
Affiliation(s)
- Giovanni Federico
- IRCCS Synlab SDN, Via Emanuele Gianturco, 113, 80143, Naples, Italy.
| | - Emanuelle Reynaud
- Laboratoire d'Etude des Mécanismes Cognitifs (EA 3082), Université de Lyon, Lyon, France
| | - Jordan Navarro
- Laboratoire d'Etude des Mécanismes Cognitifs (EA 3082), Université de Lyon, Lyon, France
| | - Mathieu Lesourd
- Laboratoire de recherches Intégratives en Neurosciences et Psychologie Cognitive (UR 481), Université de Bourgogne Franche-Comté, Besançon, France.,MSHE Ledoux, CNRS, Université de Bourgogne Franche-Comté, F-25000, Besançon, France
| | - Vivien Gaujoux
- Laboratoire d'Etude des Mécanismes Cognitifs (EA 3082), Université de Lyon, Lyon, France
| | - Franck Lamberton
- CERMEP-Imagerie du vivant, MRI Department and CNRS UMS3453, Lyon, France
| | - Danièle Ibarrola
- CERMEP-Imagerie du vivant, MRI Department and CNRS UMS3453, Lyon, France
| | - Carlo Cavaliere
- IRCCS Synlab SDN, Via Emanuele Gianturco, 113, 80143, Naples, Italy
| | - Vincenzo Alfano
- IRCCS Synlab SDN, Via Emanuele Gianturco, 113, 80143, Naples, Italy
| | - Marco Aiello
- IRCCS Synlab SDN, Via Emanuele Gianturco, 113, 80143, Naples, Italy
| | - Marco Salvatore
- IRCCS Synlab SDN, Via Emanuele Gianturco, 113, 80143, Naples, Italy
| | - Perrine Seguin
- Centre de Recherche en Neurosciences de Lyon (CRNL), Computation, Cognition and Neurophysiology Team (Inserm UMR_S 1028-CNRS-UMR 5292-Université de Lyon), Bron, France
| | - Damien Schnebelen
- Laboratoire d'Etude des Mécanismes Cognitifs (EA 3082), Université de Lyon, Lyon, France
| | | | - Yves Rossetti
- Centre de Recherche en Neurosciences de Lyon (CRNL), Trajectoires Team (Inserm UMR_S 1028-CNRS-UMR 5292-Université de Lyon), Bron, France.,Mouvement et Handicap and Neuro-Immersion, Hospices Civils de Lyon et Centre de Recherche en Neurosciences de Lyon, Hôpital Henry Gabrielle, St Genis Laval, France
| | - François Osiurak
- Laboratoire d'Etude des Mécanismes Cognitifs (EA 3082), Université de Lyon, Lyon, France.,Institut Universitaire de France, Paris, France
| |
Collapse
|
23
|
Jiang Y, Wu H, Mi Q, Zhu L. Neurocomputations of strategic behavior: From iterated to novel interactions. WIRES COGNITIVE SCIENCE 2022; 13:e1598. [PMID: 35441465 PMCID: PMC9542218 DOI: 10.1002/wcs.1598] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/10/2021] [Revised: 03/27/2022] [Accepted: 03/29/2022] [Indexed: 11/15/2022]
Abstract
Strategic interactions, where an individual's payoff depends on the decisions of multiple intelligent agents, are ubiquitous among social animals. They span a variety of important social behaviors such as competition, cooperation, coordination, and communication, and often involve complex, intertwining cognitive operations ranging from basic reward processing to higher‐order mentalization. Here, we review the progress and challenges in probing the neural and cognitive mechanisms of strategic behavior of interacting individuals, drawing an analogy to recent developments in studies of reward‐seeking behavior, in particular, how research focuses in the field of strategic behavior have been expanded from adaptive behavior based on trial‐and‐error to flexible decisions based on limited prior experience. We highlight two important research questions in the field of strategic behavior: (i) How does the brain exploit past experience for learning to behave strategically? and (ii) How does the brain decide what to do in novel strategic situations in the absence of direct experience? For the former, we discuss the utility of learning models that have effectively connected various types of neural data with strategic learning behavior and helped elucidate the interplay among multiple learning processes. For the latter, we review the recent evidence and propose a neural generative mechanism by which the brain makes novel strategic choices through simulating others' goal‐directed actions according to rational or bounded‐rational principles obtained through indirect social knowledge. This article is categorized under:Economics > Interactive Decision‐Making Psychology > Reasoning and Decision Making Neuroscience > Cognition
Collapse
Affiliation(s)
- Yaomin Jiang
- School of Psychological and Cognitive Sciences, Beijing Key Laboratory of Behavior and Mental Health, IDG/McGovern Institute for Brain Research, Peking‐Tsinghua Center for Life Sciences Peking University Beijing China
| | - Hai‐Tao Wu
- School of Psychological and Cognitive Sciences, Beijing Key Laboratory of Behavior and Mental Health, IDG/McGovern Institute for Brain Research, Peking‐Tsinghua Center for Life Sciences Peking University Beijing China
| | - Qingtian Mi
- School of Psychological and Cognitive Sciences, Beijing Key Laboratory of Behavior and Mental Health, IDG/McGovern Institute for Brain Research, Peking‐Tsinghua Center for Life Sciences Peking University Beijing China
| | - Lusha Zhu
- School of Psychological and Cognitive Sciences, Beijing Key Laboratory of Behavior and Mental Health, IDG/McGovern Institute for Brain Research, Peking‐Tsinghua Center for Life Sciences Peking University Beijing China
| |
Collapse
|
24
|
Jordan EJ, Völter CJ, Seed AM. Do capuchin monkeys ( Sapajus apella) use exploration to form intuitions about physical properties? Cogn Neuropsychol 2022; 38:531-543. [PMID: 35732407 DOI: 10.1080/02643294.2022.2088273] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/17/2022]
Abstract
Humans' flexible innovation relies on our capacity to accurately predict objects' behaviour. These predictions may originate from a "physics-engine" in the brain which simulates our environment. To explore the evolutionary origins of intuitive physics, we investigate whether capuchin monkeys' object exploration supports learning. Two capuchin groups experienced exploration sessions involving multiple copies of two objects, one object was easily opened (functional), the other was not (non-functional). We used two within-subject conditions (enrichment-then-test, and test-only) with two object sets per group. Monkeys then underwent individual test sessions where the objects contained rewards, and they choose one to attempt to open. The monkeys spontaneously explored, performing actions which yielded functional information. At test, both groups chose functional objects above chance. While high performance of the test-only group precluded us from establishing learning during exploration, this study reveals the promise of harnessing primates' natural exploratory tendencies to understand how they see the world.
Collapse
Affiliation(s)
- Eleanor Jade Jordan
- School of Psychology & Neuroscience, University of St Andrews, St Andrews, UK
| | - Christoph J Völter
- Comparative Cognition, Messerli Research Institute, University of Veterinary Medicine Vienna, Medical University of Vienna, University of Vienna, Wien, Austria
| | - Amanda M Seed
- School of Psychology & Neuroscience, University of St Andrews, St Andrews, UK
| |
Collapse
|
25
|
Knights E, Smith FW, Rossit S. The role of the anterior temporal cortex in action: evidence from fMRI multivariate searchlight analysis during real object grasping. Sci Rep 2022; 12:9042. [PMID: 35662252 PMCID: PMC9167815 DOI: 10.1038/s41598-022-12174-9] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/04/2022] [Accepted: 04/29/2022] [Indexed: 12/20/2022] Open
Abstract
Intelligent manipulation of handheld tools marks a major discontinuity between humans and our closest ancestors. Here we identified neural representations about how tools are typically manipulated within left anterior temporal cortex, by shifting a searchlight classifier through whole-brain real action fMRI data when participants grasped 3D-printed tools in ways considered typical for use (i.e., by their handle). These neural representations were automatically evocated as task performance did not require semantic processing. In fact, findings from a behavioural motion-capture experiment confirmed that actions with tools (relative to non-tool) incurred additional processing costs, as would be suspected if semantic areas are being automatically engaged. These results substantiate theories of semantic cognition that claim the anterior temporal cortex combines sensorimotor and semantic content for advanced behaviours like tool manipulation.
Collapse
Affiliation(s)
- Ethan Knights
- School of Psychology, University of East Anglia, Norwich, UK
| | - Fraser W Smith
- School of Psychology, University of East Anglia, Norwich, UK
| | | |
Collapse
|
26
|
Pramod RT, Cohen MA, Tenenbaum JB, Kanwisher N. Invariant representation of physical stability in the human brain. eLife 2022; 11:e71736. [PMID: 35635277 PMCID: PMC9150889 DOI: 10.7554/elife.71736] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2021] [Accepted: 05/10/2022] [Indexed: 11/13/2022] Open
Abstract
Successful engagement with the world requires the ability to predict what will happen next. Here, we investigate how the brain makes a fundamental prediction about the physical world: whether the situation in front of us is stable, and hence likely to stay the same, or unstable, and hence likely to change in the immediate future. Specifically, we ask if judgments of stability can be supported by the kinds of representations that have proven to be highly effective at visual object recognition in both machines and brains, or instead if the ability to determine the physical stability of natural scenes may require generative algorithms that simulate the physics of the world. To find out, we measured responses in both convolutional neural networks (CNNs) and the brain (using fMRI) to natural images of physically stable versus unstable scenarios. We find no evidence for generalizable representations of physical stability in either standard CNNs trained on visual object and scene classification (ImageNet), or in the human ventral visual pathway, which has long been implicated in the same process. However, in frontoparietal regions previously implicated in intuitive physical reasoning we find both scenario-invariant representations of physical stability, and higher univariate responses to unstable than stable scenes. These results demonstrate abstract representations of physical stability in the dorsal but not ventral pathway, consistent with the hypothesis that the computations underlying stability entail not just pattern classification but forward physical simulation.
Collapse
Affiliation(s)
- RT Pramod
- Center for Brains, Minds and Machines, Massachusetts Institute of TechnologyCambridgeUnited States
- Department of Brain and Cognitive Sciences, Massachusetts Institute of TechnologyCambridgeUnited States
| | - Michael A Cohen
- Department of Brain and Cognitive Sciences, Massachusetts Institute of TechnologyCambridgeUnited States
- Amherst CollegeAmherstUnited States
| | - Joshua B Tenenbaum
- Center for Brains, Minds and Machines, Massachusetts Institute of TechnologyCambridgeUnited States
- Department of Brain and Cognitive Sciences, Massachusetts Institute of TechnologyCambridgeUnited States
| | - Nancy Kanwisher
- Center for Brains, Minds and Machines, Massachusetts Institute of TechnologyCambridgeUnited States
- Department of Brain and Cognitive Sciences, Massachusetts Institute of TechnologyCambridgeUnited States
| |
Collapse
|
27
|
Baumard J, Lesourd M, Guézouli L, Osiurak F. Physical understanding in neurodegenerative diseases. Cogn Neuropsychol 2022; 38:490-514. [PMID: 35549825 DOI: 10.1080/02643294.2022.2071152] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
Abstract
This quantitative review gives an overview of physical understanding (i.e., the ability to represent and use the laws of physics to interact with the physical world) impairments in Alzheimer's disease (AD), semantic dementia (SD), and corticobasal syndrome (CBS), as assessed mainly with mechanical problem-solving and tool use tests. This review shows that: (1) SD patients have apraxia of tool use because of semantic tool knowledge deficits, but normal performance in tests of physical understanding; (2) AD and CBS patients show impaired performance in mechanical problem-solving tests, probably not because of intrinsic deficits of physical understanding, but rather because of additional cognitive (AD) or motor impairments (CBS); (3) As a result, the performance in mechanical problem-solving tests is not a good predictor of familiar tool use in dementia; (4) Actual deficits of physical understanding are probably observed only in late stages of neurodegenerative diseases, and associated with functional loss.
Collapse
Affiliation(s)
- Josselin Baumard
- Normandie Univ, UNIROUEN, CRFDP (EA 7475), 76000 Rouen, France.,Centre de Recherche sur les Fonctionnements et Dysfonctionnements Psychologiques (EA 7475), Mont-Saint-Aignan Cedex, France
| | - Mathieu Lesourd
- Laboratoire de Recherches Intégratives en Neurosciences et Psychologie Cognitive, Université Bourgogne Franche-Comté Besançon, France.,MSHE Ledoux, CNRS, Université de Bourgogne Franche-Comté, Besançon, France
| | - Léna Guézouli
- Normandie Univ, UNIROUEN, CRFDP (EA 7475), 76000 Rouen, France.,Centre de Recherche sur les Fonctionnements et Dysfonctionnements Psychologiques (EA 7475), Mont-Saint-Aignan Cedex, France
| | - François Osiurak
- Laboratoire d'Etude des Mécanismes Cognitifs (EA 3082), Université de Lyon, Bron Cedex, France.,Institut Universitaire de France, Paris, France
| |
Collapse
|
28
|
Hafri A, Boger T, Firestone C. Melting Ice With Your Mind: Representational Momentum for Physical States. Psychol Sci 2022; 33:725-735. [PMID: 35471852 DOI: 10.1177/09567976211051744] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Abstract
When a log burns, it transforms from a block of wood into a pile of ash. Such state changes are among the most dramatic ways objects change, going beyond mere changes of position or orientation. How does the mind represent changes of state? A foundational result in visual cognition is that memory extrapolates the positions of moving objects-a distortion called representational momentum. Here, five experiments (N = 400 adults) exploited this phenomenon to investigate mental representations in state space. Participants who viewed objects undergoing state changes (e.g., ice melting, logs burning, or grapes shriveling) remembered them as more changed (e.g., more melted, burned, or shriveled) than they actually were. This pattern extended to several types of state changes, went beyond their low-level properties, and even adhered to their natural trajectories in state space. Thus, mental representations of objects actively incorporate how they change-not only in their relation to their environment, but also in their essential qualities.
Collapse
Affiliation(s)
- Alon Hafri
- Department of Psychological & Brain Sciences, Johns Hopkins University.,Department of Cognitive Science, Johns Hopkins University
| | - Tal Boger
- Department of Psychological & Brain Sciences, Johns Hopkins University.,Department of Psychology, Yale University
| | - Chaz Firestone
- Department of Psychological & Brain Sciences, Johns Hopkins University.,Department of Cognitive Science, Johns Hopkins University.,Department of Philosophy, Johns Hopkins University
| |
Collapse
|
29
|
Navarro-Cebrián A, Fischer J. Precise functional connections between the dorsal anterior cingulate cortex and areas recruited for physical inference. Eur J Neurosci 2022; 56:3660-3673. [PMID: 35441423 PMCID: PMC9544738 DOI: 10.1111/ejn.15670] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/13/2021] [Accepted: 04/08/2022] [Indexed: 11/28/2022]
Abstract
Recent work has identified brain areas that are engaged when people predict how the physical behavior of the world will unfold - an ability termed intuitive physics. Among the many unanswered questions about the neural mechanisms of intuitive physics is where the key inputs come from: which brain regions connect up with intuitive physics processes to regulate when and how they are engaged in service of our goals? In the present work, we targeted the dorsal anterior cingulate cortex (dACC) for study based on characteristics that make it well-positioned to regulate intuitive physics processes. The dACC is richly interconnected with frontoparietal regions and is implicated in mapping contexts to actions, a process that would benefit from physical predictions to indicate which action(s) would produce the desired physical outcomes. We collected resting state functional MRI data in seventeen participants and used independent task-related runs to find the pattern of activity during a physical inference task in each individual participant. We found that the strongest resting state functional connections of the dACC not only aligned well with physical inference-related activity at the group level, it also mirrored individual differences in the positioning of physics-related activity across participants. Our results suggest that the dACC might be a key structure for regulating the engagement of intuitive physics processes in the brain.
Collapse
Affiliation(s)
- Ana Navarro-Cebrián
- Department of Psychological and Brain Sciences, Johns Hopkins University, Baltimore, MD, USA.,Department of Psychology, University of Maryland, College Park, MD, USA
| | - Jason Fischer
- Department of Psychological and Brain Sciences, Johns Hopkins University, Baltimore, MD, USA
| |
Collapse
|
30
|
Tessari A, Proietti R, Rumiati RI. Bottom-up and top-down modulation of route selection in imitation. Cogn Neuropsychol 2022; 38:515-530. [PMID: 35195056 DOI: 10.1080/02643294.2022.2043264] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Abstract
The cognitive system selects the most appropriate action imitative process: a semantic process - relying on long-term memory representations for known actions, and low-level visuomotor transformations for unknown actions. These two processes work in parallel; however, how context regularities and cognitive control modulate them is unclear. In this study, process selection was triggered contextually by presenting mixed known and new actions in predictable or unpredictable lists, while a cue on the forthcoming action triggered top-down control. Known were imitated faster than the new actions in the predictable lists only. Accuracy was higher and reaction times faster in the uncued conditions, and the predictable faster than the unpredictable list in the uncued condition only. In the latter condition, contextual factors modulate process selection, as participants use statistical regularities to perform the task at best. With the cue, the cognitive system tries to control response selection, resulting in more errors and longer reaction times.
Collapse
Affiliation(s)
| | | | - Raffaella I Rumiati
- Cognitive Neuroscience, Scuola Internazionale Superiore di Studi Avanzati, Trieste, Italy
| |
Collapse
|
31
|
Shelton AL, Davis EE, Cortesa CS, Jones JD, Hager GD, Khudanpur S, Landau B. Characterizing the Details of Spatial Construction: Cognitive Constraints and Variability. Cogn Sci 2022; 46:e13081. [DOI: 10.1111/cogs.13081] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/16/2021] [Revised: 11/23/2021] [Accepted: 11/24/2021] [Indexed: 11/30/2022]
Affiliation(s)
- Amy Lynne Shelton
- School of Education and Center for Talented Youth Johns Hopkins University
| | - E. Emory Davis
- School of Education and Center for Talented Youth Johns Hopkins University
- Department of Cognitive Science Krieger School of Arts & Sciences
| | - Cathryn S. Cortesa
- School of Education and Center for Talented Youth Johns Hopkins University
- Department of Cognitive Science Krieger School of Arts & Sciences
| | | | | | | | - Barbara Landau
- Department of Cognitive Science Krieger School of Arts & Sciences
| |
Collapse
|
32
|
Mangalam M, Fragaszy DM, Wagman JB, Day BM, Kelty-Stephen DG, Bongers RM, Stout DW, Osiurak F. On the psychological origins of tool use. Neurosci Biobehav Rev 2022; 134:104521. [PMID: 34998834 DOI: 10.1016/j.neubiorev.2022.104521] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/17/2021] [Revised: 12/01/2021] [Accepted: 01/01/2022] [Indexed: 01/13/2023]
Abstract
The ubiquity of tool use in human life has generated multiple lines of scientific and philosophical investigation to understand the development and expression of humans' engagement with tools and its relation to other dimensions of human experience. However, existing literature on tool use faces several epistemological challenges in which the same set of questions generate many different answers. At least four critical questions can be identified, which are intimately intertwined-(1) What constitutes tool use? (2) What psychological processes underlie tool use in humans and nonhuman animals? (3) Which of these psychological processes are exclusive to tool use? (4) Which psychological processes involved in tool use are exclusive to Homo sapiens? To help advance a multidisciplinary scientific understanding of tool use, six author groups representing different academic disciplines (e.g., anthropology, psychology, neuroscience) and different theoretical perspectives respond to each of these questions, and then point to the direction of future work on tool use. We find that while there are marked differences among the responses of the respective author groups to each question, there is a surprising degree of agreement about many essential concepts and questions. We believe that this interdisciplinary and intertheoretical discussion will foster a more comprehensive understanding of tool use than any one of these perspectives (or any one of these author groups) would (or could) on their own.
Collapse
Affiliation(s)
- Madhur Mangalam
- Department of Physical Therapy, Movement and Rehabilitation Science, Northeastern University, Boston, Massachusetts 02115, USA.
| | | | - Jeffrey B Wagman
- Department of Psychology, Illinois State University, Normal, IL 61761, USA
| | - Brian M Day
- Department of Psychology, Butler University, Indianapolis, IN 46208, USA
| | | | - Raoul M Bongers
- Department of Human Movement Sciences, University Medical Center Groningen, University of Groningen, 9713 GZ Groningen, Netherlands
| | - Dietrich W Stout
- Department of Anthropology, Emory University, Atlanta, GA 30322, USA
| | - François Osiurak
- Laboratoire d'Etude des Mécanismes Cognitifs, Université de Lyon, Lyon 69361, France; Institut Universitaire de France, Paris 75231, France
| |
Collapse
|
33
|
Tarhan L, De Freitas J, Konkle T. Behavioral and neural representations en route to intuitive action understanding. Neuropsychologia 2021; 163:108048. [PMID: 34653497 PMCID: PMC8649031 DOI: 10.1016/j.neuropsychologia.2021.108048] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/10/2021] [Revised: 07/13/2021] [Accepted: 10/01/2021] [Indexed: 10/20/2022]
Abstract
When we observe another person's actions, we process many kinds of information - from how their body moves to the intention behind their movements. What kinds of information underlie our intuitive understanding about how similar actions are to each other? To address this question, we measured the intuitive similarities among a large set of everyday action videos using multi-arrangement experiments, then used a modeling approach to predict this intuitive similarity space along three hypothesized properties. We found that similarity in the actors' inferred goals predicted the intuitive similarity judgments the best, followed by similarity in the actors' movements, with little contribution from the videos' visual appearance. In opportunistic fMRI analyses assessing brain-behavior correlations, we found suggestive evidence for an action processing hierarchy, in which these three kinds of action similarities are reflected in the structure of brain responses along a posterior-to-anterior gradient on the lateral surface of the visual cortex. Altogether, this work joins existing literature suggesting that humans are naturally tuned to process others' intentions, and that the visuo-motor cortex computes the perceptual precursors of the higher-level representations over which intuitive action perception operates.
Collapse
Affiliation(s)
- Leyla Tarhan
- Department of Psychology, Harvard University, USA
| | | | - Talia Konkle
- Department of Psychology, Harvard University, USA.
| |
Collapse
|
34
|
Neupärtl N, Tatai F, Rothkopf CA. Naturalistic embodied interactions elicit intuitive physical behaviour in accordance with Newtonian physics. Cogn Neuropsychol 2021; 38:440-454. [PMID: 34877918 DOI: 10.1080/02643294.2021.2008890] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Abstract
The success of visuomotor interactions in everyday activities such as grasping or sliding a cup is inescapably governed by the laws of physics. Research on intuitive physics has predominantly investigated reasoning about objects' behaviour involving binary forced choice responses. We investigated how the type of visuomotor response influences participants' beliefs about physical quantities and their lawful relationship implicit in their active behaviour. Participants propelled pucks towards targets positioned at different distances. Analysis with a probabilistic model of interactions showed that subjects adopted the non-linear control prescribed by Newtonian physics when sliding real pucks in a virtual environment even in the absence of visual feedback. However, they used a linear heuristic when viewing the scene on a monitor and interactions were implemented through key presses. These results support the notion of probabilistic internal physics models but additionally suggest that humans can take advantage of embodied, sensorimotor, multimodal representations in physical scenarios.
Collapse
Affiliation(s)
- Nils Neupärtl
- Institute of Psychology, TU Darmstadt, Darmstadt, Germany.,Centre for Cognitive Science, TU Darmstadt, Darmstadt, Germany
| | - Fabian Tatai
- Institute of Psychology, TU Darmstadt, Darmstadt, Germany.,Centre for Cognitive Science, TU Darmstadt, Darmstadt, Germany
| | - Constantin A Rothkopf
- Institute of Psychology, TU Darmstadt, Darmstadt, Germany.,Centre for Cognitive Science, TU Darmstadt, Darmstadt, Germany.,Frankfurt Institute for Advanced Studies, Goethe University, Frankfurt, Germany
| |
Collapse
|
35
|
Friston K, Moran RJ, Nagai Y, Taniguchi T, Gomi H, Tenenbaum J. World model learning and inference. Neural Netw 2021; 144:573-590. [PMID: 34634605 DOI: 10.1016/j.neunet.2021.09.011] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2021] [Revised: 07/28/2021] [Accepted: 09/09/2021] [Indexed: 11/19/2022]
Abstract
Understanding information processing in the brain-and creating general-purpose artificial intelligence-are long-standing aspirations of scientists and engineers worldwide. The distinctive features of human intelligence are high-level cognition and control in various interactions with the world including the self, which are not defined in advance and are vary over time. The challenge of building human-like intelligent machines, as well as progress in brain science and behavioural analyses, robotics, and their associated theoretical formalisations, speaks to the importance of the world-model learning and inference. In this article, after briefly surveying the history and challenges of internal model learning and probabilistic learning, we introduce the free energy principle, which provides a useful framework within which to consider neuronal computation and probabilistic world models. Next, we showcase examples of human behaviour and cognition explained under that principle. We then describe symbol emergence in the context of probabilistic modelling, as a topic at the frontiers of cognitive robotics. Lastly, we review recent progress in creating human-like intelligence by using novel probabilistic programming languages. The striking consensus that emerges from these studies is that probabilistic descriptions of learning and inference are powerful and effective ways to create human-like artificial intelligent machines and to understand intelligence in the context of how humans interact with their world.
Collapse
Affiliation(s)
- Karl Friston
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London (UCL), WC1N 3BG, UK.
| | - Rosalyn J Moran
- Department of Neuroimaging, Institute of Psychiatry, Psychology & Neuroscience, King's College London, SE5 8AF, UK.
| | - Yukie Nagai
- International Research Center for Neurointelligence (IRCN), The University of Tokyo, Tokyo, Japan.
| | - Tadahiro Taniguchi
- College of Information Science and Engineering, Ritsumeikan University, Shiga, Japan.
| | - Hiroaki Gomi
- NTT Communication Science Labs., Nippon Telegraph and Telephone, Kanawaga, Japan.
| | - Josh Tenenbaum
- Department of Brain and Cognitive Sciences, MIT, Cambridge, MA, USA; The Center for Brains, Minds and Machines, MIT, Cambridge, MA, USA.
| |
Collapse
|
36
|
Mason RA, Schumacher RA, Just MA. The neuroscience of advanced scientific concepts. NPJ SCIENCE OF LEARNING 2021; 6:29. [PMID: 34635669 PMCID: PMC8505455 DOI: 10.1038/s41539-021-00107-6] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/20/2020] [Accepted: 09/03/2021] [Indexed: 06/13/2023]
Abstract
Cognitive neuroscience methods can identify the fMRI-measured neural representation of familiar individual concepts, such as apple, and decompose them into meaningful neural and semantic components. This approach was applied here to determine the neural representations and underlying dimensions of representation of far more abstract physics concepts related to matter and energy, such as fermion and dark matter, in the brains of 10 Carnegie Mellon physics faculty members who thought about the main properties of each of the concepts. One novel dimension coded the measurability vs. immeasurability of a concept. Another novel dimension of representation evoked particularly by post-classical concepts was associated with four types of cognitive processes, each linked to particular brain regions: (1) Reasoning about intangibles, taking into account their separation from direct experience and observability; (2) Assessing consilience with other, firmer knowledge; (3) Causal reasoning about relations that are not apparent or observable; and (4) Knowledge management of a large knowledge organization consisting of a multi-level structure of other concepts. Two other underlying dimensions, previously found in physics students, periodicity, and mathematical formulation, were also present in this faculty sample. The data were analyzed using factor analysis of stably responding voxels, a Gaussian-naïve Bayes machine-learning classification of the activation patterns associated with each concept, and a regression model that predicted activation patterns associated with each concept based on independent ratings of the dimensions of the concepts. The findings indicate that the human brain systematically organizes novel scientific concepts in terms of new dimensions of neural representation.
Collapse
Affiliation(s)
- Robert A Mason
- Center for Cognitive Brain Imaging, Carnegie Mellon University, Pittsburgh, PA, 15213, USA.
- Department of Psychology, Carnegie Mellon University, Pittsburgh, PA, 15213, USA.
| | | | - Marcel Adam Just
- Center for Cognitive Brain Imaging, Carnegie Mellon University, Pittsburgh, PA, 15213, USA
- Department of Psychology, Carnegie Mellon University, Pittsburgh, PA, 15213, USA
| |
Collapse
|
37
|
Fischer J, Mahon BZ. What tool representation, intuitive physics, and action have in common: The brain's first-person physics engine. Cogn Neuropsychol 2021; 38:455-467. [PMID: 35994054 DOI: 10.1080/02643294.2022.2106126] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2022] [Revised: 07/17/2022] [Accepted: 07/21/2022] [Indexed: 10/15/2022]
Abstract
An overlapping set of brain regions in parietal and frontal cortex are engaged by different types of tasks and stimuli: (i) making inferences about the physical structure and dynamics of the world, (ii) passively viewing, or actively interacting with, manipulable objects, and (iii) planning and execution of reaching and grasping actions. We suggest the observed neural overlap is because a common superordinate computation is engaged by each of those different tasks: A forward model of physical reasoning about how first-person actions will affect the world and be affected by unfolding physical events. This perspective offers an account of why some physical predictions are systematically incorrect - there can be a mismatch between how physical scenarios are experimentally framed and the native format of the inferences generated by the brain's first-person physics engine. This perspective generates new empirical expectations about the conditions under which physical reasoning may exhibit systematic biases.
Collapse
Affiliation(s)
- Jason Fischer
- Department of Psychological and Brain Sciences, Johns Hopkins University, Baltimore, MD, USA
| | - Bradford Z Mahon
- Department of Psychology, Carnegie Mellon University, Pittsburgh, PA, USA
- Carnegie Mellon Neuroscience Institute, Pittsburgh, PA, USA
| |
Collapse
|
38
|
Ahuja A, Desrochers TM, Sheinberg DL. A role for visual areas in physics simulations. Cogn Neuropsychol 2021; 38:425-439. [PMID: 35156547 PMCID: PMC9374848 DOI: 10.1080/02643294.2022.2034609] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/06/2021] [Revised: 10/28/2021] [Accepted: 01/22/2022] [Indexed: 10/19/2022]
Abstract
To engage with the world, we must regularly make predictions about the outcomes of physical scenes. How do we make these predictions? Recent computational evidence points to simulation-the idea that we can introspectively manipulate rich, mental models of the world-as one explanation for how such predictions are accomplished. However, questions about the potential neural mechanisms of simulation remain. We hypothesized that the process of simulating physical events would evoke imagery-like representations in visual areas of those same events. Using functional magnetic resonance imaging, we find that when participants are asked to predict the likely trajectory of a falling ball, motion-sensitive brain regions are activated. We demonstrate that this activity, which occurs even though no motion is being sensed, resembles activity patterns that arise while participants perceive the ball's motion. This finding thus suggests that mental simulations recreate sensory depictions of how a physical scene is likely to unfold.
Collapse
Affiliation(s)
- Aarit Ahuja
- Department of Neuroscience, Brown University, Providence, RI, USA
| | - Theresa M Desrochers
- Department of Neuroscience, Brown University, Providence, RI, USA
- Robert J. and Nancy D. Carney Institute for Brain Science, Brown University, Providence, RI, USA
- Department of Psychiatry and Human Behavior, Brown University, Providence, RI, USA
| | - David L Sheinberg
- Department of Neuroscience, Brown University, Providence, RI, USA
- Robert J. and Nancy D. Carney Institute for Brain Science, Brown University, Providence, RI, USA
| |
Collapse
|
39
|
Stout D. The Cognitive Science of Technology. Trends Cogn Sci 2021; 25:964-977. [PMID: 34362661 DOI: 10.1016/j.tics.2021.07.005] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2021] [Revised: 07/09/2021] [Accepted: 07/13/2021] [Indexed: 01/23/2023]
Abstract
Technology is central to human life but hard to define and study. This review synthesizes advances in fields from anthropology to evolutionary biology and neuroscience to propose an interdisciplinary cognitive science of technology. The foundation of this effort is an evolutionarily motivated definition of technology that highlights three key features: material production, social collaboration, and cultural reproduction. This broad scope respects the complexity of the subject but poses a challenge for theoretical unification. Addressing this challenge requires a comparative approach to reduce the diversity of real-world technological cognition to a smaller number of recurring processes and relationships. To this end, a synthetic perceptual-motor hypothesis (PMH) for the evolutionary-developmental-cultural construction of technological cognition is advanced as an initial target for investigation.
Collapse
Affiliation(s)
- Dietrich Stout
- Department of Anthropology, Emory University, 1557 Dickey Drive, Atlanta, GA 30322, USA.
| |
Collapse
|
40
|
Fedorenko E. The early origins and the growing popularity of the individual-subject analytic approach in human neuroscience. Curr Opin Behav Sci 2021. [DOI: 10.1016/j.cobeha.2021.02.023] [Citation(s) in RCA: 26] [Impact Index Per Article: 8.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
|
41
|
Wang L, Li M, Yang T, Wang L, Zhou X. Mathematics Meets Science in the Brain. Cereb Cortex 2021; 32:123-136. [PMID: 34247249 DOI: 10.1093/cercor/bhab198] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2021] [Revised: 06/03/2021] [Accepted: 06/05/2021] [Indexed: 01/24/2023] Open
Abstract
Mathematics and science are highly integrated disciplines, but the brain association between mathematics and science remains unclear. The current study used functional magnetic resonance imaging (fMRI) scans of 34 undergraduates (17 males, mean age = 20.3±1.64 years old) while they completed mathematical, physical and chemical principles, arithmetic computation, and sentence comprehension. We examined neural activation level, neural activation pattern, and neural connectivity to investigate the neural associations between mathematics and science (including physics and chemistry). The results showed that mathematical, physical, and chemical principles elicited similar neural activation level and neural activation pattern in the visuospatial network (mainly in the middle frontal gyrus and inferior parietal lobule), which were different from those elicited by sentence comprehension; those three principles also elicited similar neural activation level and neural activation pattern in the semantic network (mainly in the middle temporal gyrus, angular gyrus, inferior frontal gyrus, and dorsomedial prefrontal cortex), in contrast to that elicited by arithmetic computation. Effective connectivity analyses showed stronger connectivity between the middle temporal gyrus and inferior parietal lobule for mathematical, physical, and chemical principles than for sentence comprehension. The results suggest that visuospatial and semantic networks were critical for processing both mathematics and science.
Collapse
Affiliation(s)
- Li Wang
- State Key Laboratory of Cognitive Neuroscience and Learning, Faculty of Psychology, Beijing Normal University, Beijing 100875, China.,Advanced Innovation Center for Future Education, Beijing Normal University, Beijing 102206, China.,Siegler center for Innovative Learning, Beijing Normal University, Beijing 100875, China.,Center for Brain and Mathematical learning, Beijing Normal University, Beijing 100875, China
| | - Mengyi Li
- State Key Laboratory of Cognitive Neuroscience and Learning, Faculty of Psychology, Beijing Normal University, Beijing 100875, China.,Advanced Innovation Center for Future Education, Beijing Normal University, Beijing 102206, China.,Siegler center for Innovative Learning, Beijing Normal University, Beijing 100875, China.,Center for Brain and Mathematical learning, Beijing Normal University, Beijing 100875, China
| | - Tao Yang
- Collaborative Innovation Center of Assessment for Basic Education Quality, Beijing Normal University, Beijing 100875, China
| | - Li Wang
- Collaborative Innovation Center of Assessment for Basic Education Quality, Beijing Normal University, Beijing 100875, China
| | - Xinlin Zhou
- State Key Laboratory of Cognitive Neuroscience and Learning, Faculty of Psychology, Beijing Normal University, Beijing 100875, China.,Advanced Innovation Center for Future Education, Beijing Normal University, Beijing 102206, China.,Siegler center for Innovative Learning, Beijing Normal University, Beijing 100875, China.,Center for Brain and Mathematical learning, Beijing Normal University, Beijing 100875, China
| |
Collapse
|
42
|
Knights E, Mansfield C, Tonin D, Saada J, Smith FW, Rossit S. Hand-Selective Visual Regions Represent How to Grasp 3D Tools: Brain Decoding during Real Actions. J Neurosci 2021; 41:5263-5273. [PMID: 33972399 PMCID: PMC8211542 DOI: 10.1523/jneurosci.0083-21.2021] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2021] [Revised: 03/23/2021] [Accepted: 03/29/2021] [Indexed: 02/02/2023] Open
Abstract
Most neuroimaging experiments that investigate how tools and their actions are represented in the brain use visual paradigms where tools or hands are displayed as 2D images and no real movements are performed. These studies discovered selective visual responses in occipitotemporal and parietal cortices for viewing pictures of hands or tools, which are assumed to reflect action processing, but this has rarely been directly investigated. Here, we examined the responses of independently visually defined category-selective brain areas when participants grasped 3D tools (N = 20; 9 females). Using real-action fMRI and multivoxel pattern analysis, we found that grasp typicality representations (i.e., whether a tool is grasped appropriately for use) were decodable from hand-selective areas in occipitotemporal and parietal cortices, but not from tool-, object-, or body-selective areas, even if partially overlapping. Importantly, these effects were exclusive for actions with tools, but not for biomechanically matched actions with control nontools. In addition, grasp typicality decoding was significantly higher in hand than tool-selective parietal regions. Notably, grasp typicality representations were automatically evoked even when there was no requirement for tool use and participants were naive to object category (tool vs nontools). Finding a specificity for typical tool grasping in hand-selective, rather than tool-selective, regions challenges the long-standing assumption that activation for viewing tool images reflects sensorimotor processing linked to tool manipulation. Instead, our results show that typicality representations for tool grasping are automatically evoked in visual regions specialized for representing the human hand, the primary tool of the brain for interacting with the world.
Collapse
Affiliation(s)
- Ethan Knights
- Medical Research Council Cognition and Brain Sciences Unit, University of Cambridge, Cambridge CB2 7EF, United Kingdom
| | - Courtney Mansfield
- School of Psychology, University of East Anglia, Norwich NR4 7TJ, United Kingdom
| | - Diana Tonin
- School of Psychology, University of East Anglia, Norwich NR4 7TJ, United Kingdom
| | - Janak Saada
- Department of Radiology, Norfolk and Norwich University Hospitals NHS Foundation Trust, Norwich NR4 7UY, United Kingdom
| | - Fraser W Smith
- School of Psychology, University of East Anglia, Norwich NR4 7TJ, United Kingdom
| | - Stéphanie Rossit
- School of Psychology, University of East Anglia, Norwich NR4 7TJ, United Kingdom
| |
Collapse
|
43
|
Vélez N, Gweon H. Learning from other minds: an optimistic critique of reinforcement learning models of social learning. Curr Opin Behav Sci 2021; 38:110-115. [DOI: 10.1016/j.cobeha.2021.01.006] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/26/2023]
|
44
|
Ivanova AA, Mineroff Z, Zimmerer V, Kanwisher N, Varley R, Fedorenko E. The Language Network Is Recruited but Not Required for Nonverbal Event Semantics. NEUROBIOLOGY OF LANGUAGE (CAMBRIDGE, MASS.) 2021; 2:176-201. [PMID: 37216147 PMCID: PMC10158592 DOI: 10.1162/nol_a_00030] [Citation(s) in RCA: 23] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/04/2020] [Accepted: 01/07/2021] [Indexed: 05/24/2023]
Abstract
The ability to combine individual concepts of objects, properties, and actions into complex representations of the world is often associated with language. Yet combinatorial event-level representations can also be constructed from nonverbal input, such as visual scenes. Here, we test whether the language network in the human brain is involved in and necessary for semantic processing of events presented nonverbally. In Experiment 1, we scanned participants with fMRI while they performed a semantic plausibility judgment task versus a difficult perceptual control task on sentences and line drawings that describe/depict simple agent-patient interactions. We found that the language network responded robustly during the semantic task performed on both sentences and pictures (although its response to sentences was stronger). Thus, language regions in healthy adults are engaged during a semantic task performed on pictorial depictions of events. But is this engagement necessary? In Experiment 2, we tested two individuals with global aphasia, who have sustained massive damage to perisylvian language areas and display severe language difficulties, against a group of age-matched control participants. Individuals with aphasia were severely impaired on the task of matching sentences to pictures. However, they performed close to controls in assessing the plausibility of pictorial depictions of agent-patient interactions. Overall, our results indicate that the left frontotemporal language network is recruited but not necessary for semantic processing of nonverbally presented events.
Collapse
Affiliation(s)
- Anna A. Ivanova
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Zachary Mineroff
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Vitor Zimmerer
- Division of Psychology and Language Sciences, University College London, London, UK
| | - Nancy Kanwisher
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Rosemary Varley
- Division of Psychology and Language Sciences, University College London, London, UK
| | - Evelina Fedorenko
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA, USA
| |
Collapse
|
45
|
Fazeli N, Oller M, Wu J, Wu Z, Tenenbaum JB, Rodriguez A. See, feel, act: Hierarchical learning for complex manipulation skills with multisensory fusion. Sci Robot 2021; 4:4/26/eaav3123. [PMID: 33137764 DOI: 10.1126/scirobotics.aav3123] [Citation(s) in RCA: 24] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/05/2018] [Accepted: 01/04/2019] [Indexed: 11/02/2022]
Abstract
Humans are able to seamlessly integrate tactile and visual stimuli with their intuitions to explore and execute complex manipulation skills. They not only see but also feel their actions. Most current robotic learning methodologies exploit recent progress in computer vision and deep learning to acquire data-hungry pixel-to-action policies. These methodologies do not exploit intuitive latent structure in physics or tactile signatures. Tactile reasoning is omnipresent in the animal kingdom, yet it is underdeveloped in robotic manipulation. Tactile stimuli are only acquired through invasive interaction, and interpretation of the data stream together with visual stimuli is challenging. Here, we propose a methodology to emulate hierarchical reasoning and multisensory fusion in a robot that learns to play Jenga, a complex game that requires physical interaction to be played effectively. The game mechanics were formulated as a generative process using a temporal hierarchical Bayesian model, with representations for both behavioral archetypes and noisy block states. This model captured descriptive latent structures, and the robot learned probabilistic models of these relationships in force and visual domains through a short exploration phase. Once learned, the robot used this representation to infer block behavior patterns and states as it played the game. Using its inferred beliefs, the robot adjusted its behavior with respect to both its current actions and its game strategy, similar to the way humans play the game. We evaluated the performance of the approach against three standard baselines and show its fidelity on a real-world implementation of the game.
Collapse
Affiliation(s)
- N Fazeli
- Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA.
| | - M Oller
- Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - J Wu
- Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Z Wu
- Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - J B Tenenbaum
- Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - A Rodriguez
- Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| |
Collapse
|
46
|
How do the object-file and physical-reasoning systems interact? Evidence from priming effects with object arrays or novel labels. Cogn Psychol 2021; 125:101368. [PMID: 33421683 DOI: 10.1016/j.cogpsych.2020.101368] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2020] [Revised: 12/18/2020] [Accepted: 12/19/2020] [Indexed: 11/22/2022]
Abstract
How do infants reason about simple physical events such as containment, tube, and support events? According to the two-system model, two cognitive systems, the object-file (OF) and physical-reasoning (PR) systems, work together to guide infants' responses to these events. When an event begins, the OF system sends categorical information about the objects and their arrangements to the PR system. This system then categorizes the event, assigns event roles to the objects, and taps the OF system for information about features previously identified as causally relevant for the event category selected. All of the categorical and featural information included in the event's representation is interpreted by the PR system's domain knowledge, which includes core principles such as persistence and gravity. The present research tested a novel prediction of the model: If the OF system could be primed to also send, at the beginning of an event, information about an as-yet-unidentified feature, the PR system would then interpret this information using its core principles, allowing infants to detect core violations involving the feature earlier than they normally would. We examined this prediction using two types of priming manipulations directed at the OF system, object arrays and novel labels. In six experiments, infants aged 7-13 months (N = 304) were tested using different event categories and as-yet-unidentified features (color in containment events, height in tube events, and proportional distribution in support events) as well as different tasks (violation-of-expectation and action tasks). In each case, infants who were effectively primed reasoned successfully about the as-yet-unidentified feature, sometimes as early as six months before they would typically do so. These converging results provide strong support for the two-system model and for the claim that uncovering how the OF and PR systems represent and exchange information is essential for understanding how infants respond to physical events.
Collapse
|
47
|
Abstract
Humanity has regarded itself as intellectually superior to other species for millennia, yet human cognitive uniqueness remains poorly understood. Here, we evaluate candidate traits plausibly underlying our distinctive cognition (including mental time travel, tool use, problem solving, social cognition, and communication) as well as domain generality, and we consider how human cognitive uniqueness may have evolved. We conclude that there are no traits present in humans and absent in other animals that in isolation explain our species' superior cognitive performance; rather, there are many cognitive domains in which humans possess unusually potent capabilities compared to those found in other species. Humans are flexible cognitive all-rounders, whose proficiency arises through interactions and reinforcement between cognitive domains at multiple scales.
Collapse
Affiliation(s)
- Kevin Laland
- School of Biology, University of St. Andrews, St. Andrews KY16 9ST, United Kingdom;
| | - Amanda Seed
- School of Psychology and Neuroscience, University of St. Andrews, St. Andrews KY16 9JP, United Kingdom
| |
Collapse
|
48
|
Ivanova AA, Srikant S, Sueoka Y, Kean HH, Dhamala R, O'Reilly UM, Bers MU, Fedorenko E. Comprehension of computer code relies primarily on domain-general executive brain regions. eLife 2020; 9:e58906. [PMID: 33319744 PMCID: PMC7738192 DOI: 10.7554/elife.58906] [Citation(s) in RCA: 20] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2020] [Accepted: 11/06/2020] [Indexed: 12/22/2022] Open
Abstract
Computer programming is a novel cognitive tool that has transformed modern society. What cognitive and neural mechanisms support this skill? Here, we used functional magnetic resonance imaging to investigate two candidate brain systems: the multiple demand (MD) system, typically recruited during math, logic, problem solving, and executive tasks, and the language system, typically recruited during linguistic processing. We examined MD and language system responses to code written in Python, a text-based programming language (Experiment 1) and in ScratchJr, a graphical programming language (Experiment 2); for both, we contrasted responses to code problems with responses to content-matched sentence problems. We found that the MD system exhibited strong bilateral responses to code in both experiments, whereas the language system responded strongly to sentence problems, but weakly or not at all to code problems. Thus, the MD system supports the use of novel cognitive tools even when the input is structurally similar to natural language.
Collapse
Affiliation(s)
- Anna A Ivanova
- Department of Brain and Cognitive Sciences, Massachusetts Institute of TechnologyCambridgeUnited States
- McGovern Institute for Brain Research, Massachusetts Institute of TechnologyCambridgeUnited States
| | - Shashank Srikant
- Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of TechnologyCambridgeUnited States
| | - Yotaro Sueoka
- Department of Brain and Cognitive Sciences, Massachusetts Institute of TechnologyCambridgeUnited States
- McGovern Institute for Brain Research, Massachusetts Institute of TechnologyCambridgeUnited States
| | - Hope H Kean
- Department of Brain and Cognitive Sciences, Massachusetts Institute of TechnologyCambridgeUnited States
- McGovern Institute for Brain Research, Massachusetts Institute of TechnologyCambridgeUnited States
| | - Riva Dhamala
- Eliot-Pearson Department of Child Study and Human Development, Tufts UniversityMedfordUnited States
| | - Una-May O'Reilly
- Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of TechnologyCambridgeUnited States
| | - Marina U Bers
- Eliot-Pearson Department of Child Study and Human Development, Tufts UniversityMedfordUnited States
| | - Evelina Fedorenko
- Department of Brain and Cognitive Sciences, Massachusetts Institute of TechnologyCambridgeUnited States
- McGovern Institute for Brain Research, Massachusetts Institute of TechnologyCambridgeUnited States
| |
Collapse
|
49
|
Josephs EL, Konkle T. Large-scale dissociations between views of objects, scenes, and reachable-scale environments in visual cortex. Proc Natl Acad Sci U S A 2020; 117:29354-29362. [PMID: 33229533 PMCID: PMC7703543 DOI: 10.1073/pnas.1912333117] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022] Open
Abstract
Space-related processing recruits a network of brain regions separate from those recruited in object processing. This dissociation has largely been explored by contrasting views of navigable-scale spaces to views of close-up, isolated objects. However, in naturalistic visual experience, we encounter spaces intermediate to these extremes, like the tops of desks and kitchen counters, which are not navigable but typically contain multiple objects. How are such reachable-scale views represented in the brain? In three human functional neuroimaging experiments, we find evidence for a large-scale dissociation of reachable-scale views from both navigable scene views and close-up object views. Three brain regions were identified that showed a systematic response preference to reachable views, located in the posterior collateral sulcus, the inferior parietal sulcus, and superior parietal lobule. Subsequent analyses suggest that these three regions may be especially sensitive to the presence of multiple objects. Further, in all classic scene and object regions, reachable-scale views dissociated from both objects and scenes with an intermediate response magnitude. Taken together, these results establish that reachable-scale environments have a distinct representational signature from both scene and object views in visual cortex.
Collapse
Affiliation(s)
- Emilie L Josephs
- Department of Psychology, Harvard University, Cambridge, MA 02138
| | - Talia Konkle
- Department of Psychology, Harvard University, Cambridge, MA 02138
| |
Collapse
|
50
|
Freiwald WA. Social interaction networks in the primate brain. Curr Opin Neurobiol 2020; 65:49-58. [PMID: 33065333 DOI: 10.1016/j.conb.2020.08.012] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/21/2020] [Revised: 08/27/2020] [Accepted: 08/31/2020] [Indexed: 12/23/2022]
Abstract
Primate brains have evolved to understand and engage with their social world. Much about the structure of this world can be gleaned from social interactions. Circuits for the analysis of and participation in social interactions have now been mapped. Increased knowledge about their functional specializations and relative spatial locations promises to greatly improve the understanding of the functional organization of the primate social brain. Detailed electrophysiology, as in the case of the face-processing network, of local operations and functional interactions between areas is necessary to uncover neural mechanisms and computation principles of social cognition. New naturalistic behavioral paradigms, behavioral tracking, and new analytical approaches for parallel non-stationary data will be important components toward a neuroscientific theory of primates' interactive minds.
Collapse
Affiliation(s)
- Winrich A Freiwald
- The Rockefeller University, New York, United States; Center for Brains, Minds, and Machines, United States.
| |
Collapse
|