1
|
Perdikis S, Leeb R, Chavarriaga R, Millan JDR. Context-Aware Learning for Generative Models. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2021; 32:3471-3483. [PMID: 32776882 DOI: 10.1109/tnnls.2020.3011671] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
This work studies the class of algorithms for learning with side-information that emerges by extending generative models with embedded context-related variables. Using finite mixture models (FMMs) as the prototypical Bayesian network, we show that maximum-likelihood estimation (MLE) of parameters through expectation-maximization (EM) improves over the regular unsupervised case and can approach the performances of supervised learning, despite the absence of any explicit ground-truth data labeling. By direct application of the missing information principle (MIP), the algorithms' performances are proven to range between the conventional supervised and unsupervised MLE extremities proportionally to the information content of the contextual assistance provided. The acquired benefits regard higher estimation precision, smaller standard errors, faster convergence rates, and improved classification accuracy or regression fitness shown in various scenarios while also highlighting important properties and differences among the outlined situations. Applicability is showcased with three real-world unsupervised classification scenarios employing Gaussian mixture models. Importantly, we exemplify the natural extension of this methodology to any type of generative model by deriving an equivalent context-aware algorithm for variational autoencoders (VAs), thus broadening the spectrum of applicability to unsupervised deep learning with artificial neural networks. The latter is contrasted with a neural-symbolic algorithm exploiting side information.
Collapse
|
2
|
Fiksel J, Datta A, Amouzou A, Zeger S. Generalized Bayes Quantification Learning under Dataset Shift. J Am Stat Assoc 2021. [DOI: 10.1080/01621459.2021.1909599] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
Affiliation(s)
- Jacob Fiksel
- Department of Biostatistics, Johns Hopkins University, Baltimore, MD
| | - Abhirup Datta
- Department of Biostatistics, Johns Hopkins University, Baltimore, MD
| | - Agbessi Amouzou
- Department of International Health, Johns Hopkins University, Baltimore, MD
| | - Scott Zeger
- Department of Biostatistics, Johns Hopkins University, Baltimore, MD
| |
Collapse
|
3
|
Franke K, Berens P, Schubert T, Bethge M, Euler T, Baden T. Inhibition decorrelates visual feature representations in the inner retina. Nature 2017; 542:439-444. [PMID: 28178238 PMCID: PMC5325673 DOI: 10.1038/nature21394] [Citation(s) in RCA: 145] [Impact Index Per Article: 20.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2016] [Accepted: 01/18/2017] [Indexed: 01/25/2023]
Abstract
The retina extracts visual features for transmission to the brain. Different types of bipolar cell split the photoreceptor input into parallel channels and provide the excitatory drive for downstream visual circuits. Anatomically and genetically, mouse bipolar cell types have been described at great detail, but a similarly deep understanding of their functional diversity is lacking. By imaging light-driven glutamate release from more than 13,000 bipolar cell axon terminals in the intact retina, we here show that bipolar cell functional diversity is generated by the interplay of dendritic excitatory inputs and axonal inhibitory inputs. The resultant centre and surround components of bipolar cell receptive fields interact to decorrelate bipolar cell output in the spatial and temporal domain. Our findings highlight the importance of inhibitory circuits in generating functionally diverse excitatory pathways and suggest that decorrelation of parallel visual pathways begins already at the second synapse of the mouse visual system.
Collapse
Affiliation(s)
- Katrin Franke
- Centre for Integrative Neuroscience, University of Tübingen, Tübingen, Germany.,Bernstein Centre for Computational Neuroscience, University of Tübingen, Tübingen, Germany.,Institute for Ophthalmic Research, University of Tübingen, Tübingen, Germany.,Graduate School of Neural &Behavioural Sciences, International Max Planck Research School, University of Tübingen, Tübingen, Germany
| | - Philipp Berens
- Centre for Integrative Neuroscience, University of Tübingen, Tübingen, Germany.,Bernstein Centre for Computational Neuroscience, University of Tübingen, Tübingen, Germany.,Institute for Ophthalmic Research, University of Tübingen, Tübingen, Germany
| | - Timm Schubert
- Centre for Integrative Neuroscience, University of Tübingen, Tübingen, Germany.,Institute for Ophthalmic Research, University of Tübingen, Tübingen, Germany
| | - Matthias Bethge
- Centre for Integrative Neuroscience, University of Tübingen, Tübingen, Germany.,Bernstein Centre for Computational Neuroscience, University of Tübingen, Tübingen, Germany.,Institute for Theoretical Physics, University of Tübingen, Tübingen, Germany.,Max Planck Institute of Biological Cybernetics, Tübingen, Germany
| | - Thomas Euler
- Centre for Integrative Neuroscience, University of Tübingen, Tübingen, Germany.,Bernstein Centre for Computational Neuroscience, University of Tübingen, Tübingen, Germany.,Institute for Ophthalmic Research, University of Tübingen, Tübingen, Germany
| | - Tom Baden
- Centre for Integrative Neuroscience, University of Tübingen, Tübingen, Germany.,Bernstein Centre for Computational Neuroscience, University of Tübingen, Tübingen, Germany.,Institute for Ophthalmic Research, University of Tübingen, Tübingen, Germany.,School of Life Sciences, University of Sussex, Brighton, UK
| |
Collapse
|
4
|
Marsico A, Huska MR, Lasserre J, Hu H, Vucicevic D, Musahl A, Orom U, Vingron M. PROmiRNA: a new miRNA promoter recognition method uncovers the complex regulation of intronic miRNAs. Genome Biol 2013; 14:R84. [PMID: 23958307 PMCID: PMC4053815 DOI: 10.1186/gb-2013-14-8-r84] [Citation(s) in RCA: 89] [Impact Index Per Article: 8.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2013] [Accepted: 08/16/2013] [Indexed: 12/21/2022] Open
Abstract
The regulation of intragenic miRNAs by their own intronic promoters is one of the open problems of miRNA biogenesis. Here, we describe PROmiRNA, a new approach for miRNA promoter annotation based on a semi-supervised statistical model trained on deepCAGE data and sequence features. We validate our results with existing annotation, PolII occupancy data and read coverage from RNA-seq data. Compared to previous methods PROmiRNA increases the detection rate of intronic promoters by 30%, allowing us to perform a large-scale analysis of their genomic features, as well as elucidate their contribution to tissue-specific regulation. PROmiRNA can be downloaded from http://promirna.molgen.mpg.de.
Collapse
|
5
|
Szczurek E, Markowetz F, Gat-Viks I, Biecek P, Tiuryn J, Vingron M. Deregulation upon DNA damage revealed by joint analysis of context-specific perturbation data. BMC Bioinformatics 2011; 12:249. [PMID: 21693013 PMCID: PMC3236061 DOI: 10.1186/1471-2105-12-249] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2010] [Accepted: 06/21/2011] [Indexed: 12/17/2022] Open
Abstract
Background Deregulation between two different cell populations manifests itself in changing gene expression patterns and changing regulatory interactions. Accumulating knowledge about biological networks creates an opportunity to study these changes in their cellular context. Results We analyze re-wiring of regulatory networks based on cell population-specific perturbation data and knowledge about signaling pathways and their target genes. We quantify deregulation by merging regulatory signal from the two cell populations into one score. This joint approach, called JODA, proves advantageous over separate analysis of the cell populations and analysis without incorporation of knowledge. JODA is implemented and freely available in a Bioconductor package 'joda'. Conclusions Using JODA, we show wide-spread re-wiring of gene regulatory networks upon neocarzinostatin-induced DNA damage in Human cells. We recover 645 deregulated genes in thirteen functional clusters performing the rich program of response to damage. We find that the clusters contain many previously characterized neocarzinostatin target genes. We investigate connectivity between those genes, explaining their cooperation in performing the common functions. We review genes with the most extreme deregulation scores, reporting their involvement in response to DNA damage. Finally, we investigate the indirect impact of the ATM pathway on the deregulated genes, and build a hypothetical hierarchy of direct regulation. These results prove that JODA is a step forward to a systems level, mechanistic understanding of changes in gene regulation between different cell populations.
Collapse
Affiliation(s)
- Ewa Szczurek
- Computational Molecular Biology Department, Max Planck Institute for Molecular Genetics, Ihnestrasse 73, 14195 Berlin, Germany.
| | | | | | | | | | | |
Collapse
|