1
|
Dai H, Pollock M, Roberts GO. Bayesian fusion: scalable unification of distributed statistical analyses. J R Stat Soc Series B Stat Methodol 2023. [DOI: 10.1093/jrsssb/qkac007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/03/2023]
Abstract
Abstract
There has been considerable interest in addressing the problem of unifying distributed analyses into a single coherent inference, which arises in big-data settings, when working under privacy constraints, and in Bayesian model choice. Most existing approaches relied upon approximations of the distributed analyses, which have significant shortcomings—the quality of the inference can degrade rapidly with the number of analyses being unified, and can be substantially biased when unifying analyses that do not concur. In contrast, recent Monte Carlo fusion approach is exact and based on rejection sampling. In this paper, we introduce a practical Bayesian fusion approach by embedding the Monte Carlo fusion framework within a sequential Monte Carlo algorithm. We demonstrate theoretically and empirically that Bayesian fusion is more robust than existing methods.
Collapse
Affiliation(s)
- Hongsheng Dai
- Department of Mathematical Sciences, University of Essex , Colchester , UK
| | - Murray Pollock
- School of Mathematics, Statistics and Physics, Newcastle University , Newcastle-upon-Tyne , UK
- The Alan Turing Institute , London , UK
| | - Gareth O Roberts
- The Alan Turing Institute , London , UK
- Department of Statistics, University of Warwick , Coventry , UK
| |
Collapse
|
2
|
Sen D. Particle filter efficiency under limited communication. Biometrika 2022. [DOI: 10.1093/biomet/asac015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022] Open
Abstract
Summary
Sequential Monte Carlo methods are typically not straightforward to implement on parallel architectures. This is because standard resampling schemes involve communication between all particles. The α-sequential Monte Carlo method was proposed recently as a potential solution to this which limits communication between particles. This limited communication is controlled through a sequence of stochastic matrices known as α-matrices. We study the influence of the communication structure on the convergence and stability properties of the resulting algorithms. In particular, we quantitatively show that the mixing properties of the α-matrices play an important role in the stability properties of the algorithm. Moreover, we prove that one can ensure good mixing properties by using randomized communication structures where each particle only communicates with a few neighbouring particles. The resulting algorithms converge at the usual Monte Carlo rate. This leads to efficient versions of distributed sequential Monte Carlo.
Collapse
Affiliation(s)
- Deborshee Sen
- Department of Mathematical Sciences, University of Bath, Claverton Down, Bath BA2 7AY, U.K
| |
Collapse
|
3
|
Heine K, Whiteley N, Cemgil A. Parallelizing particle filters with butterfly interactions. Scand Stat Theory Appl 2019. [DOI: 10.1111/sjos.12408] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
Affiliation(s)
- Kari Heine
- Department of Mathematical SciencesUniversity of Bath Bath UK
| | - Nick Whiteley
- School of MathematicsUniversity of Bristol Bristol UK
| | - A.Taylan Cemgil
- Department of Computer EngineeringBoğaziçi University Istanbul Turkey
| |
Collapse
|