1
|
Friston KJ, Parr T, Heins C, Constant A, Friedman D, Isomura T, Fields C, Verbelen T, Ramstead M, Clippinger J, Frith CD. Federated inference and belief sharing. Neurosci Biobehav Rev 2024; 156:105500. [PMID: 38056542 DOI: 10.1016/j.neubiorev.2023.105500] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2023] [Revised: 11/08/2023] [Accepted: 12/01/2023] [Indexed: 12/08/2023]
Abstract
This paper concerns the distributed intelligence or federated inference that emerges under belief-sharing among agents who share a common world-and world model. Imagine, for example, several animals keeping a lookout for predators. Their collective surveillance rests upon being able to communicate their beliefs-about what they see-among themselves. But, how is this possible? Here, we show how all the necessary components arise from minimising free energy. We use numerical studies to simulate the generation, acquisition and emergence of language in synthetic agents. Specifically, we consider inference, learning and selection as minimising the variational free energy of posterior (i.e., Bayesian) beliefs about the states, parameters and structure of generative models, respectively. The common theme-that attends these optimisation processes-is the selection of actions that minimise expected free energy, leading to active inference, learning and model selection (a.k.a., structure learning). We first illustrate the role of communication in resolving uncertainty about the latent states of a partially observed world, on which agents have complementary perspectives. We then consider the acquisition of the requisite language-entailed by a likelihood mapping from an agent's beliefs to their overt expression (e.g., speech)-showing that language can be transmitted across generations by active learning. Finally, we show that language is an emergent property of free energy minimisation, when agents operate within the same econiche. We conclude with a discussion of various perspectives on these phenomena; ranging from cultural niche construction, through federated learning, to the emergence of complexity in ensembles of self-organising systems.
Collapse
Affiliation(s)
- Karl J Friston
- Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, UK; VERSES AI Research Lab, Los Angeles, CA 90016, USA.
| | - Thomas Parr
- Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, UK
| | - Conor Heins
- VERSES AI Research Lab, Los Angeles, CA 90016, USA; Department of Collective Behaviour, Max Planck Institute of Animal Behavior, 78457 Konstanz, Germany; Centre for the Advanced Study of Collective Behaviour, 78457 Konstanz, Germany; Department of Biology, University of Konstanz, 78457 Konstanz, Germany
| | - Axel Constant
- VERSES AI Research Lab, Los Angeles, CA 90016, USA; School of Engineering and Informatics, The University of Sussex, Brighton, UK
| | - Daniel Friedman
- Department of Entomology and Nematology, University of California, Davis, Davis, CA, USA; Active Inference Institute, Davis, CA 95616, USA
| | - Takuya Isomura
- Brain Intelligence Theory Unit, RIKEN Center for Brain Science, Wako, Saitama 351-0198, Japan
| | - Chris Fields
- Allen Discovery Center at Tufts University, Medford, MA 02155, USA
| | - Tim Verbelen
- VERSES AI Research Lab, Los Angeles, CA 90016, USA
| | - Maxwell Ramstead
- Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, UK; VERSES AI Research Lab, Los Angeles, CA 90016, USA
| | | | - Christopher D Frith
- Institute of Philosophy, School of Advanced Studies, University of London, UK
| |
Collapse
|
2
|
Shen X, Pan S, Choi KS, Zhou X. Domain-adaptive message passing graph neural network. Neural Netw 2023; 164:439-454. [PMID: 37182346 DOI: 10.1016/j.neunet.2023.04.038] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2022] [Revised: 04/18/2023] [Accepted: 04/21/2023] [Indexed: 05/16/2023]
Abstract
Cross-network node classification (CNNC), which aims to classify nodes in a label-deficient target network by transferring the knowledge from a source network with abundant labels, draws increasing attention recently. To address CNNC, we propose a domain-adaptive message passing graph neural network (DM-GNN), which integrates graph neural network (GNN) with conditional adversarial domain adaptation. DM-GNN is capable of learning informative representations for node classification that are also transferrable across networks. Firstly, a GNN encoder is constructed by dual feature extractors to separate ego-embedding learning from neighbor-embedding learning so as to jointly capture commonality and discrimination between connected nodes. Secondly, a label propagation node classifier is proposed to refine each node's label prediction by combining its own prediction and its neighbors' prediction. In addition, a label-aware propagation scheme is devised for the labeled source network to promote intra-class propagation while avoiding inter-class propagation, thus yielding label-discriminative source embeddings. Thirdly, conditional adversarial domain adaptation is performed to take the neighborhood-refined class-label information into account during adversarial domain adaptation, so that the class-conditional distributions across networks can be better matched. Comparisons with eleven state-of-the-art methods demonstrate the effectiveness of the proposed DM-GNN.
Collapse
Affiliation(s)
- Xiao Shen
- School of Computer Science and Technology, Hainan University, Haikou, China.
| | - Shirui Pan
- School of ICT, Griffith University, Gold Coast, Australia.
| | - Kup-Sze Choi
- Centre for Smart Health, The Hong Kong Polytechnic University, Hong Kong, China.
| | - Xi Zhou
- College of Tropical Crops, Hainan University, Haikou, China.
| |
Collapse
|
3
|
Li X, Cheng Y. Understanding the message passing in graph neural networks via power iteration clustering. Neural Netw 2021; 140:130-5. [PMID: 33765528 DOI: 10.1016/j.neunet.2021.02.025] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2020] [Revised: 01/11/2021] [Accepted: 02/22/2021] [Indexed: 11/22/2022]
Abstract
The mechanism of message passing in graph neural networks (GNNs) is still mysterious. Apart from convolutional neural networks, no theoretical origin for GNNs has been proposed. To our surprise, message passing can be best understood in terms of power iteration. By fully or partly removing activation functions and layer weights of GNNs, we propose subspace power iteration clustering (SPIC) models that iteratively learn with only one aggregator. Experiments show that our models extend GNNs and enhance their capability to process random featured networks. Moreover, we demonstrate the redundancy of some state-of-the-art GNNs in design and define a lower limit for model evaluation by a random aggregator of message passing. Our findings push the boundaries of the theoretical understanding of neural networks.
Collapse
|
4
|
Friston KJ, Parr T, Yufik Y, Sajid N, Price CJ, Holmes E. Generative models, linguistic communication and active inference. Neurosci Biobehav Rev 2020; 118:42-64. [PMID: 32687883 PMCID: PMC7758713 DOI: 10.1016/j.neubiorev.2020.07.005] [Citation(s) in RCA: 30] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/25/2020] [Revised: 06/26/2020] [Accepted: 07/08/2020] [Indexed: 11/24/2022]
Abstract
This paper presents a biologically plausible generative model and inference scheme that is capable of simulating communication between synthetic subjects who talk to each other. Building on active inference formulations of dyadic interactions, we simulate linguistic exchange to explore generative models that support dialogues. These models employ high-order interactions among abstract (discrete) states in deep (hierarchical) models. The sequential nature of language processing mandates generative models with a particular factorial structure-necessary to accommodate the rich combinatorics of language. We illustrate linguistic communication by simulating a synthetic subject who can play the 'Twenty Questions' game. In this game, synthetic subjects take the role of the questioner or answerer, using the same generative model. This simulation setup is used to illustrate some key architectural points and demonstrate that many behavioural and neurophysiological correlates of linguistic communication emerge under variational (marginal) message passing, given the right kind of generative model. For example, we show that theta-gamma coupling is an emergent property of belief updating, when listening to another.
Collapse
Affiliation(s)
- Karl J Friston
- The Wellcome Centre for Human Neuroimaging, UCL Queen Square Institute of Neurology, 12 Queen Square, London, WC1N 3AR, UK.
| | - Thomas Parr
- The Wellcome Centre for Human Neuroimaging, UCL Queen Square Institute of Neurology, 12 Queen Square, London, WC1N 3AR, UK.
| | - Yan Yufik
- Virtual Structures Research, Inc., 12204 Saint James Rd, Potomac, MD 20854, USA.
| | - Noor Sajid
- The Wellcome Centre for Human Neuroimaging, UCL Queen Square Institute of Neurology, 12 Queen Square, London, WC1N 3AR, UK.
| | - Catherine J Price
- The Wellcome Centre for Human Neuroimaging, UCL Queen Square Institute of Neurology, 12 Queen Square, London, WC1N 3AR, UK.
| | - Emma Holmes
- The Wellcome Centre for Human Neuroimaging, UCL Queen Square Institute of Neurology, 12 Queen Square, London, WC1N 3AR, UK.
| |
Collapse
|
5
|
Zhao S, Wu X, Chen B, Li S. Automatic vertebrae recognition from arbitrary spine MRI images by a category-Consistent self-calibration detection framework. Med Image Anal 2020; 67:101826. [PMID: 33075638 DOI: 10.1016/j.media.2020.101826] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2019] [Revised: 09/28/2020] [Accepted: 09/29/2020] [Indexed: 11/30/2022]
Abstract
Accurate vertebrae recognition is crucial in spinal disease localization and successive treatment planning. Although vertebrae detection has been studied for years, reliably recognizing vertebrae from arbitrary spine MRI images remains a challenge. The similar appearance of different vertebrae and the pathological deformations of the same vertebrae makes it difficult for classification in images with different fields of view (FOV). In this paper, we propose a Category-consistent Self-calibration Recognition System (Can-See) to accurately classify the labels and precisely predict the bounding boxes of all vertebrae with improved discriminative capabilities for vertebrae categories and self-awareness of false positive detections. Can-See is designed as a two-step detection framework: (1) A hierarchical proposal network (HPN) to perceive the existence of the vertebrae. HPN leverages the correspondence between hierarchical features and multi-scale anchors to detect objects. This correspondence tackles the image scale/resolution challenge. (2) A Category-consistent Self-calibration Recognition (CSRN) Network to classify each vertebra and refine their bounding boxes. CSRN leverages the dictionary learning principle to preserve the most representative features; it imposes a novel category-consistent constraint to force vertebrae with the same label to have similar features. CSRN then innovatively formulates message passing into the deep learning framework, which leverages the label compatibility principle to self-calibrate the wrong pre-recognitions. Can-See is trained and evaluated on a capacious and challenging dataset of 450 MRI scans. The results show that Can-See achieves high performance (testing accuracy reaches 0.955) and outperforms other state-of-the-art methods.
Collapse
Affiliation(s)
- Shen Zhao
- Sun Yat-Sen University, Guangzhou, China
| | - Xi Wu
- Department of Computer Science, Chengdu University of Information Technology, Chengdu,610225,China.
| | - Bo Chen
- University of Western Ontario, London ON, Canada
| | - Shuo Li
- University of Western Ontario, London ON, Canada.
| |
Collapse
|
6
|
Abstract
This paper considers functional integration in the brain from a computational perspective. We ask what sort of neuronal message passing is mandated by active inference—and what implications this has for context-sensitive connectivity at microscopic and macroscopic levels. In particular, we formulate neuronal processing as belief propagation under deep generative models. Crucially, these models can entertain both discrete and continuous states, leading to distinct schemes for belief updating that play out on the same (neuronal) architecture. Technically, we use Forney (normal) factor graphs to elucidate the requisite message passing in terms of its form and scheduling. To accommodate mixed generative models (of discrete and continuous states), one also has to consider link nodes or factors that enable discrete and continuous representations to talk to each other. When mapping the implicit computational architecture onto neuronal connectivity, several interesting features emerge. For example, Bayesian model averaging and comparison, which link discrete and continuous states, may be implemented in thalamocortical loops. These and other considerations speak to a computational connectome that is inherently state dependent and self-organizing in ways that yield to a principled (variational) account. We conclude with simulations of reading that illustrate the implicit neuronal message passing, with a special focus on how discrete (semantic) representations inform, and are informed by, continuous (visual) sampling of the sensorium. This paper considers functional integration in the brain from a computational perspective. We ask what sort of neuronal message passing is mandated by active inference—and what implications this has for context-sensitive connectivity at microscopic and macroscopic levels. In particular, we formulate neuronal processing as belief propagation under deep generative models that can entertain both discrete and continuous states. This leads to distinct schemes for belief updating that play out on the same (neuronal) architecture. Technically, we use Forney (normal) factor graphs to characterize the requisite message passing, and link this formal characterization to canonical microcircuits and extrinsic connectivity in the brain.
Collapse
Affiliation(s)
- Karl J Friston
- Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, United Kingdom
| | - Thomas Parr
- Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, United Kingdom
| | - Bert de Vries
- Eindhoven University of Technology, Department of Electrical Engineering, Eindhoven, The Netherlands.,GN Hearing, Eindhoven, The Netherlands
| |
Collapse
|
7
|
Wilkinson RR, Ball FG, Sharkey KJ. The relationships between message passing, pairwise, Kermack-McKendrick and stochastic SIR epidemic models. J Math Biol 2017; 75:1563-1590. [PMID: 28409223 PMCID: PMC5641366 DOI: 10.1007/s00285-017-1123-8] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2016] [Revised: 02/16/2017] [Indexed: 11/28/2022]
Abstract
We consider a very general stochastic model for an SIR epidemic on a network which allows an individual's infectious period, and the time it takes to contact each of its neighbours after becoming infected, to be correlated. We write down the message passing system of equations for this model and prove, for the first time, that it has a unique feasible solution. We also generalise an earlier result by proving that this solution provides a rigorous upper bound for the expected epidemic size (cumulative number of infection events) at any fixed time [Formula: see text]. We specialise these results to a homogeneous special case where the graph (network) is symmetric. The message passing system here reduces to just four equations. We prove that cycles in the network inhibit the spread of infection, and derive important epidemiological results concerning the final epidemic size and threshold behaviour for a major outbreak. For Poisson contact processes, this message passing system is equivalent to a non-Markovian pair approximation model, which we show has well-known pairwise models as special cases. We show further that a sequence of message passing systems, starting with the homogeneous one just described, converges to the deterministic Kermack-McKendrick equations for this stochastic model. For Poisson contact and recovery, we show that this convergence is monotone, from which it follows that the message passing system (and hence also the pairwise model) here provides a better approximation to the expected epidemic size at time [Formula: see text] than the Kermack-McKendrick model.
Collapse
Affiliation(s)
| | - Frank G. Ball
- School of Mathematical Sciences, University of Nottingham, University Park, Nottingham, NG7 2RD UK
| | | |
Collapse
|
8
|
González-Domínguez J, Remeseiro B, Martín MJ. Parallel definition of tear film maps on distributed-memory clusters for the support of dry eye diagnosis. Comput Methods Programs Biomed 2017; 139:51-60. [PMID: 28187895 DOI: 10.1016/j.cmpb.2016.10.027] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/01/2016] [Revised: 09/20/2016] [Accepted: 10/24/2016] [Indexed: 06/06/2023]
Abstract
BACKGROUND AND OBJECTIVES The analysis of the interference patterns on the tear film lipid layer is a useful clinical test to diagnose dry eye syndrome. This task can be automated with a high degree of accuracy by means of the use of tear film maps. However, the time required by the existing applications to generate them prevents a wider acceptance of this method by medical experts. Multithreading has been previously successfully employed by the authors to accelerate the tear film map definition on multicore single-node machines. In this work, we propose a hybrid message-passing and multithreading parallel approach that further accelerates the generation of tear film maps by exploiting the computational capabilities of distributed-memory systems such as multicore clusters and supercomputers. METHODS The algorithm for drawing tear film maps is parallelized using Message Passing Interface (MPI) for inter-node communications and the multithreading support available in the C++11 standard for intra-node parallelization. The original algorithm is modified to reduce the communications and increase the scalability. RESULTS The hybrid method has been tested on 32 nodes of an Intel cluster (with two 12-core Haswell 2680v3 processors per node) using 50 representative images. Results show that maximum runtime is reduced from almost two minutes using the previous only-multithreaded approach to less than ten seconds using the hybrid method. CONCLUSIONS The hybrid MPI/multithreaded implementation can be used by medical experts to obtain tear film maps in only a few seconds, which will significantly accelerate and facilitate the diagnosis of the dry eye syndrome.
Collapse
Affiliation(s)
- Jorge González-Domínguez
- Grupo de Arquitectura de Computadores, Universidade da Coruña, Campus de Elviña s/n, 15071 A Coruña, Spain.
| | - Beatriz Remeseiro
- INESC TEC-INESC Technology and Science, Campus da FEUP, Rua Dr. Roberto Frias, 4200-465 Porto, Portugal
| | - María J Martín
- Grupo de Arquitectura de Computadores, Universidade da Coruña, Campus de Elviña s/n, 15071 A Coruña, Spain
| |
Collapse
|
9
|
Abstract
UNLABELLED This paper considers functional integration in the brain from a computational perspective. We ask what sort of neuronal message passing is mandated by active inference-and what implications this has for context-sensitive connectivity at microscopic and macroscopic levels. In particular, we formulate neuronal processing as belief propagation under deep generative models. Crucially, these models can entertain both discrete and continuous states, leading to distinct schemes for belief updating that play out on the same (neuronal) architecture. Technically, we use Forney (normal) factor graphs to elucidate the requisite message passing in terms of its form and scheduling. To accommodate mixed generative models (of discrete and continuous states), one also has to consider link nodes or factors that enable discrete and continuous representations to talk to each other. When mapping the implicit computational architecture onto neuronal connectivity, several interesting features emerge. For example, Bayesian model averaging and comparison, which link discrete and continuous states, may be implemented in thalamocortical loops. These and other considerations speak to a computational connectome that is inherently state dependent and self-organizing in ways that yield to a principled (variational) account. We conclude with simulations of reading that illustrate the implicit neuronal message passing, with a special focus on how discrete (semantic) representations inform, and are informed by, continuous (visual) sampling of the sensorium. AUTHOR SUMMARY This paper considers functional integration in the brain from a computational perspective. We ask what sort of neuronal message passing is mandated by active inference-and what implications this has for context-sensitive connectivity at microscopic and macroscopic levels. In particular, we formulate neuronal processing as belief propagation under deep generative models that can entertain both discrete and continuous states. This leads to distinct schemes for belief updating that play out on the same (neuronal) architecture. Technically, we use Forney (normal) factor graphs to characterize the requisite message passing, and link this formal characterization to canonical microcircuits and extrinsic connectivity in the brain.
Collapse
Affiliation(s)
- Karl J Friston
- Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, United Kingdom
| | - Thomas Parr
- Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, United Kingdom
| | - Bert de Vries
- Eindhoven University of Technology, Department of Electrical Engineering, Eindhoven, The Netherlands
- GN Hearing, Eindhoven, The Netherlands
| |
Collapse
|