1
|
Chase EC, Taylor JMG, Boonstra PS. Modeling basal body temperature data using horseshoe process regression. Stat Med 2024; 43:817-832. [PMID: 38095078 DOI: 10.1002/sim.9991] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2022] [Revised: 08/07/2023] [Accepted: 12/03/2023] [Indexed: 02/21/2024]
Abstract
Biomedical data often exhibit jumps or abrupt changes. For example, women's basal body temperature may jump at ovulation, menstruation, implantation, and miscarriage. These sudden changes make these data challenging to model: many methods will oversmooth the sharp changes or overfit in response to measurement error. We develop horseshoe process regression (HPR) to address this problem. We define a horseshoe process as a stochastic process in which each increment is horseshoe-distributed. We use the horseshoe process as a nonparametric Bayesian prior for modeling a potentially nonlinear association between an outcome and its continuous predictor, which we implement via Stan and in the R package HPR. We provide guidance and extensions to advance HPR's use in applied practice: we introduce a Bayesian imputation scheme to allow for interpolation at unobserved values of the predictor within the HPR; include additional covariates via a partial linear model framework; and allow for monotonicity constraints. We find that HPR performs well when fitting functions that have sharp changes. We apply HPR to model women's basal body temperatures over the course of the menstrual cycle.
Collapse
Affiliation(s)
| | - Jeremy M G Taylor
- Department of Biostatistics, University of Michigan, Ann Arbor, Michigan, USA
| | - Philip S Boonstra
- Department of Biostatistics, University of Michigan, Ann Arbor, Michigan, USA
| |
Collapse
|
2
|
Martin GM, Frazier DT, Robert CP. Approximating Bayes in the 21st Century. Stat Sci 2023. [DOI: 10.1214/22-sts875] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/25/2023]
Affiliation(s)
- Gael M. Martin
- Gael M. Martin is Professor, Department of Econometrics and Business Statistics, Monash University, Melbourne, Australia
| | - David T. Frazier
- David T. Frazier is Associate Professor, Department of Econometrics and Business Statistics, Monash University, Melbourne, Australia
| | | |
Collapse
|
3
|
Hughes DM, García-Fiñana M, Wand MP. Fast approximate inference for multivariate longitudinal data. Biostatistics 2022; 24:177-192. [PMID: 33991420 DOI: 10.1093/biostatistics/kxab021] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2020] [Revised: 04/23/2021] [Accepted: 04/27/2021] [Indexed: 12/16/2022] Open
Abstract
Collecting information on multiple longitudinal outcomes is increasingly common in many clinical settings. In many cases, it is desirable to model these outcomes jointly. However, in large data sets, with many outcomes, computational burden often prevents the simultaneous modeling of multiple outcomes within a single model. We develop a mean field variational Bayes algorithm, to jointly model multiple Gaussian, Poisson, or binary longitudinal markers within a multivariate generalized linear mixed model. Through simulation studies and clinical applications (in the fields of sight threatening diabetic retinopathy and primary biliary cirrhosis), we demonstrate substantial computational savings of our approximate approach when compared to a standard Markov Chain Monte Carlo, while maintaining good levels of accuracy of model parameters.
Collapse
Affiliation(s)
- David M Hughes
- Department of Health Data Science, Waterhouse Building, Block F, University of Liverpool, 1-5 Brownlow Street, Liverpool, L69 3GL, UK
| | - Marta García-Fiñana
- Department of Health Data Science, Waterhouse Building, Block F, University of Liverpool, 1-5 Brownlow Street, Liverpool, L69 3GL, UK
| | - Matt P Wand
- School of Mathematical and Physical Sciences, University of Technology Sydney, P.O. Box 123, Broadway, NSW 2007, AUSTRALIA
| |
Collapse
|
4
|
Menictas M, Di Credico G, Wand MP. Streamlined Variational Inference for Linear Mixed Models with Crossed Random Effects. J Comput Graph Stat 2022; 32:99-115. [PMID: 36873962 PMCID: PMC9983814 DOI: 10.1080/10618600.2022.2096622] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2021] [Accepted: 06/20/2022] [Indexed: 10/17/2022]
Abstract
We derive streamlined mean field variational Bayes algorithms for fitting linear mixed models with crossed random effects. In the most general situation, where the dimensions of the crossed groups are arbitrarily large, streamlining is hindered by lack of sparseness in the underlying least squares system. Because of this fact we also consider a hierarchy of relaxations of the mean field product restriction. The least stringent product restriction delivers a high degree of inferential accuracy. However, this accuracy must be mitigated against its higher storage and computing demands. Faster sparse storage and computing alternatives are also provided, but come with the price of diminished inferential accuracy. This article provides full algorithmic details of three variational inference strategies, presents detailed empirical results on their pros and cons and, thus, guides the users on their choice of variational inference approach depending on the problem size and computing resources.
Collapse
|
5
|
Abstract
Bayesian estimation of multidimensional item response theory (IRT) models in large data sets may come with impractical computational burdens when general-purpose Markov chain Monte Carlo (MCMC) samplers are employed. Variational Bayes (VB)—a method for approximating the posterior distribution—poses a potential remedy. Stan’s general-purpose VB algorithms have drastically improved the accessibility of VB methods for a wide psychometric audience. Using marginal maximum likelihood (MML) and MCMC as benchmarks, the present simulation study investigates the utility of Stan’s built-in VB function for estimating multidimensional IRT models with between-item dimensionality. VB yielded a marked speed-up in comparison to MCMC, but did not generally outperform MML in terms of run time. VB estimates were trustworthy only for item difficulties, while bias in item discriminations depended on the model’s dimensionality. Under realistic conditions of non-zero correlations between dimensions, VB correlation estimates were subject to severe bias. The practical relevance of performance differences is illustrated with data from PISA 2018. We conclude that in its current form, Stan’s built-in VB algorithm does not pose a viable alternative for estimating multidimensional IRT models.
Collapse
|
6
|
Maestrini L, Wand MP. The Inverse G‐Wishart distribution and variational message passing. AUST NZ J STAT 2021. [DOI: 10.1111/anzs.12339] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
Affiliation(s)
- Luca Maestrini
- School of Mathematical and Physical Sciences University of Technology Sydney P.O. Box 123, Broadway NSW2007Australia
| | - Matt P. Wand
- School of Mathematical and Physical Sciences University of Technology Sydney P.O. Box 123, Broadway NSW2007Australia
| |
Collapse
|
7
|
Şenöz İ, van de Laar T, Bagaev D, de Vries B. Variational Message Passing and Local Constraint Manipulation in Factor Graphs. ENTROPY 2021; 23:e23070807. [PMID: 34202913 PMCID: PMC8303273 DOI: 10.3390/e23070807] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/19/2021] [Revised: 06/18/2021] [Accepted: 06/22/2021] [Indexed: 11/30/2022]
Abstract
Accurate evaluation of Bayesian model evidence for a given data set is a fundamental problem in model development. Since evidence evaluations are usually intractable, in practice variational free energy (VFE) minimization provides an attractive alternative, as the VFE is an upper bound on negative model log-evidence (NLE). In order to improve tractability of the VFE, it is common to manipulate the constraints in the search space for the posterior distribution of the latent variables. Unfortunately, constraint manipulation may also lead to a less accurate estimate of the NLE. Thus, constraint manipulation implies an engineering trade-off between tractability and accuracy of model evidence estimation. In this paper, we develop a unifying account of constraint manipulation for variational inference in models that can be represented by a (Forney-style) factor graph, for which we identify the Bethe Free Energy as an approximation to the VFE. We derive well-known message passing algorithms from first principles, as the result of minimizing the constrained Bethe Free Energy (BFE). The proposed method supports evaluation of the BFE in factor graphs for model scoring and development of new message passing-based inference algorithms that potentially improve evidence estimation accuracy.
Collapse
Affiliation(s)
- İsmail Şenöz
- Department of Electrical Engineering, Eindhoven University of Technology, 5600 MB Eindhoven, The Netherlands; (T.v.d.L.); (D.B.); (B.d.V.)
- Correspondence:
| | - Thijs van de Laar
- Department of Electrical Engineering, Eindhoven University of Technology, 5600 MB Eindhoven, The Netherlands; (T.v.d.L.); (D.B.); (B.d.V.)
| | - Dmitry Bagaev
- Department of Electrical Engineering, Eindhoven University of Technology, 5600 MB Eindhoven, The Netherlands; (T.v.d.L.); (D.B.); (B.d.V.)
| | - Bert de Vries
- Department of Electrical Engineering, Eindhoven University of Technology, 5600 MB Eindhoven, The Netherlands; (T.v.d.L.); (D.B.); (B.d.V.)
- GN Hearing, JF Kennedylaan 2, 5612 AB Eindhoven, The Netherlands
| |
Collapse
|
8
|
Srivastava S, Xu Y. Distributed Bayesian Inference in Linear Mixed-Effects Models. J Comput Graph Stat 2021. [DOI: 10.1080/10618600.2020.1869025] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
Affiliation(s)
- Sanvesh Srivastava
- Department of Statistics and Actuarial Science, The University of Iowa, Iowa City, IA
| | - Yixiang Xu
- Marketing Group, Haas School of Business, University of California, Berkeley, Berkeley, CA
| |
Collapse
|
9
|
Laplace approximations for fast Bayesian inference in generalized additive models based on P-splines. Comput Stat Data Anal 2021. [DOI: 10.1016/j.csda.2020.107088] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
|
10
|
|
11
|
Durante D, Rigon T. Conditionally Conjugate Mean-Field Variational Bayes for Logistic Models. Stat Sci 2019. [DOI: 10.1214/19-sts712] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
|
12
|
Fasiolo M, Nedellec R, Goude Y, Wood SN. Scalable Visualization Methods for Modern Generalized Additive Models. J Comput Graph Stat 2019. [DOI: 10.1080/10618600.2019.1629942] [Citation(s) in RCA: 24] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
Affiliation(s)
- Matteo Fasiolo
- School of Mathematics, University of Bristol, Bristol, UK
| | | | | | - Simon N. Wood
- School of Mathematics, University of Bristol, Bristol, UK
| |
Collapse
|
13
|
Cox M, van de Laar T, de Vries B. A factor graph approach to automated design of Bayesian signal processing algorithms. Int J Approx Reason 2019. [DOI: 10.1016/j.ijar.2018.11.002] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/21/2023]
|
14
|
Affiliation(s)
- Luca Maestrini
- Department of Statistical Sciences; University of Padua; Via Cesare Battisti 241 35121 Padua PD Italy
| | - Matt P. Wand
- School of Mathematical and Physical Sciences; University of Technology Sydney; PO Box 123, Broadway 2007 NSW Australia
| |
Collapse
|
15
|
Kim AS, Wand MP. On expectation propagation for generalised, linear and mixed models. AUST NZ J STAT 2018. [DOI: 10.1111/anzs.12199] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Affiliation(s)
- Andy S.I. Kim
- University of Technology Sydney and Australian Research Council Centre of Excellence for Mathematical and Statistical Frontiers
- School of Mathematical and Physical Sciences; University of Technology Sydney; Broadway 2007 Australia
| | - Matt P. Wand
- University of Technology Sydney and Australian Research Council Centre of Excellence for Mathematical and Statistical Frontiers
- School of Mathematical and Physical Sciences; University of Technology Sydney; Broadway 2007 Australia
| |
Collapse
|
16
|
Tran D, Blei DM. Comment. J Am Stat Assoc 2017. [DOI: 10.1080/01621459.2016.1270044] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Affiliation(s)
- Dustin Tran
- Department of Computer Science and Statistics, Columbia University, New York, NY
| | - David M. Blei
- Department of Computer Science and Statistics, Columbia University, New York, NY
| |
Collapse
|