1
|
Garcia-Angulo AC, Claeskens G. Optimal finite sample post-selection confidence distributions in generalized linear models. J Stat Plan Inference 2023. [DOI: 10.1016/j.jspi.2022.06.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
|
2
|
Gao LL, Bien J, Witten D. Selective Inference for Hierarchical Clustering. J Am Stat Assoc 2022; 119:332-342. [PMID: 38660582 PMCID: PMC11036349 DOI: 10.1080/01621459.2022.2116331] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2020] [Accepted: 08/16/2022] [Indexed: 10/17/2022]
Abstract
Classical tests for a difference in means control the type I error rate when the groups are defined a priori. However, when the groups are instead defined via clustering, then applying a classical test yields an extremely inflated type I error rate. Notably, this problem persists even if two separate and independent data sets are used to define the groups and to test for a difference in their means. To address this problem, in this paper, we propose a selective inference approach to test for a difference in means between two clusters. Our procedure controls the selective type I error rate by accounting for the fact that the choice of null hypothesis was made based on the data. We describe how to efficiently compute exact p-values for clusters obtained using agglomerative hierarchical clustering with many commonly-used linkages. We apply our method to simulated data and to single-cell RNA-sequencing data.
Collapse
Affiliation(s)
- Lucy L. Gao
- Department of Statistics, University of British Columbia
| | - Jacob Bien
- Department of Data Sciences and Operations, University of Southern California
| | - Daniela Witten
- Departments of Statistics and Biostatistics, University of Washington
| |
Collapse
|
3
|
Chen Y, Jewell S, Witten D. More Powerful Selective Inference for the Graph Fused Lasso. J Comput Graph Stat 2022; 32:577-587. [PMID: 38250478 PMCID: PMC10798806 DOI: 10.1080/10618600.2022.2097246] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2021] [Accepted: 06/28/2022] [Indexed: 10/17/2022]
Abstract
The graph fused lasso-which includes as a special case the one-dimensional fused lasso-is widely used to reconstruct signals that are piecewise constant on a graph, meaning that nodes connected by an edge tend to have identical values. We consider testing for a difference in the means of two connected components estimated using the graph fused lasso. A naive procedure such as a z-test for a difference in means will not control the selective Type I error, since the hypothesis that we are testing is itself a function of the data. In this work, we propose a new test for this task that controls the selective Type I error, and conditions on less information than existing approaches, leading to substantially higher power. We illustrate our approach in simulation and on datasets of drug overdose death rates and teenage birth rates in the contiguous United States. Our approach yields more discoveries on both datasets. Supplementary materials for this article are available online.
Collapse
Affiliation(s)
- Yiqun Chen
- Department of Biostatistics, University of Washington, Seattle, WA
| | - Sean Jewell
- Department of Statistics, University of Washington, Seattle, WA
| | - Daniela Witten
- Department of Biostatistics, University of Washington, Seattle, WA
- Department of Statistics, University of Washington, Seattle, WA
| |
Collapse
|
4
|
Selective inference after feature selection via multiscale bootstrap. ANN I STAT MATH 2022. [DOI: 10.1007/s10463-022-00838-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/01/2022]
|
5
|
Zhang Y, Politis DN. Ridge regression revisited: Debiasing, thresholding and bootstrap. Ann Stat 2022. [DOI: 10.1214/21-aos2156] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
Affiliation(s)
- Yunyi Zhang
- Department of Mathematics, University of California, San Diego
| | - Dimitris N. Politis
- Department of Mathematics and Halicioglu Data Science Institute, University of California, San Diego
| |
Collapse
|
6
|
Garcia‐Angulo AC, Claeskens G. Exact uniformly most powerful post‐selection confidence distributions. Scand Stat Theory Appl 2022. [DOI: 10.1111/sjos.12581] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
Affiliation(s)
- Andrea C. Garcia‐Angulo
- ORStat and Leuven Statistics Research Centre KU Leuven
- Facultad de Ciencias Naturales y Matemáticas, Escuela Superior Politécnica del Litoral, ESPOL
| | | |
Collapse
|
7
|
Rügamer D, Baumann PF, Greven S. Selective inference for additive and linear mixed models. Comput Stat Data Anal 2022. [DOI: 10.1016/j.csda.2021.107350] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/03/2022]
|
8
|
Zhang D, Khalili A, Asgharian M. Post-model-selection inference in linear regression models: An integrated review. STATISTICS SURVEYS 2022. [DOI: 10.1214/22-ss135] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
Affiliation(s)
- Dongliang Zhang
- Department of Biostatistics, Johns Hopkins University, Baltimore, MD, USA
| | - Abbas Khalili
- Department of Mathematics and Statistics, McGill University, Montréal, QC, Canada
| | - Masoud Asgharian
- Department of Mathematics and Statistics, McGill University, Montréal, QC, Canada
| |
Collapse
|
9
|
Yuen C, Fryzlewicz P. Exploiting Disagreement Between High-Dimensional Variable Selectors for Uncertainty Visualization. J Comput Graph Stat 2021. [DOI: 10.1080/10618600.2021.2000421] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Affiliation(s)
- Christine Yuen
- Department of Statistics, London School of Economics and Political Science, London, UK
| | - Piotr Fryzlewicz
- Department of Statistics, London School of Economics and Political Science, London, UK
| |
Collapse
|
10
|
Rathnayake RC, Olive DJ. Bootstrapping some GLM and survival regression variable selection estimators. COMMUN STAT-THEOR M 2021. [DOI: 10.1080/03610926.2021.1955389] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
Affiliation(s)
- Rasanji C. Rathnayake
- School of Mathematical & Statistical Sciences, Southern Illinois University, Carbondale, Illinois, USA
| | - David J. Olive
- School of Mathematical & Statistical Sciences, Southern Illinois University, Carbondale, Illinois, USA
| |
Collapse
|
11
|
Zheng Z, Liu L, Li Y, Zhao N. High-dimensional statistical inference via DATE. COMMUN STAT-THEOR M 2021. [DOI: 10.1080/03610926.2021.1909733] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
Affiliation(s)
- Zemin Zheng
- School of Management, University of Science and Technology of China, Hefei, Anhui, P. R. China
| | - Lei Liu
- School of Management, University of Science and Technology of China, Hefei, Anhui, P. R. China
| | - Yang Li
- School of Management, University of Science and Technology of China, Hefei, Anhui, P. R. China
| | - Ni Zhao
- School of Mathematics and Physics Sciences, Anhui Jianzhu University, Hefei, Anhui, P. R. China
| |
Collapse
|
12
|
Hyun S, Lin KZ, G'Sell M, Tibshirani RJ. Post-selection inference for changepoint detection algorithms with application to copy number variation data. Biometrics 2021; 77:1037-1049. [PMID: 33434289 DOI: 10.1111/biom.13422] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2019] [Revised: 07/06/2020] [Accepted: 08/11/2020] [Indexed: 11/30/2022]
Abstract
Changepoint detection methods are used in many areas of science and engineering, for example, in the analysis of copy number variation data to detect abnormalities in copy numbers along the genome. Despite the broad array of available tools, methodology for quantifying our uncertainty in the strength (or the presence) of given changepoints post-selection are lacking. Post-selection inference offers a framework to fill this gap, but the most straightforward application of these methods results in low-powered hypothesis tests and leaves open several important questions about practical usability. In this work, we carefully tailor post-selection inference methods toward changepoint detection, focusing on copy number variation data. To accomplish this, we study commonly used changepoint algorithms: binary segmentation, as well as two of its most popular variants, wild and circular, and the fused lasso. We implement some of the latest developments in post-selection inference theory, mainly auxiliary randomization. This improves the power, which requires implementations of Markov chain Monte Carlo algorithms (importance sampling and hit-and-run sampling) to carry out our tests. We also provide recommendations for improving practical useability, detailed simulations, and example analyses on array comparative genomic hybridization as well as sequencing data.
Collapse
Affiliation(s)
- Sangwon Hyun
- Department of Data Sciences and Operations, University of Southern California, Los Angeles, California, USA
| | - Kevin Z Lin
- Department of Statistics, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Max G'Sell
- Department of Statistics and Data Science, Carnegie Mellon University, Pittsburgh, Pennsylvania, USA
| | - Ryan J Tibshirani
- Department of Statistics and Data Science, Carnegie Mellon University, Pittsburgh, Pennsylvania, USA
| |
Collapse
|
13
|
Watanabe C, Suzuki T. Selective inference for latent block models. Electron J Stat 2021. [DOI: 10.1214/21-ejs1853] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
Affiliation(s)
- Chihiro Watanabe
- Graduate School of Information Science Technology, The University of Tokyo, Tokyo, Japan
| | - Taiji Suzuki
- Graduate School of Information Science Technology, The University of Tokyo, Tokyo, Japan
| |
Collapse
|
14
|
Schultheiss C, Renaux C, Bühlmann P. Multicarving for high-dimensional post-selection inference. Electron J Stat 2021. [DOI: 10.1214/21-ejs1825] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
|
15
|
Kuchibhotla AK, Brown LD, Buja A, Cai J, George EI, Zhao LH. Valid post-selection inference in model-free linear regression. Ann Stat 2020. [DOI: 10.1214/19-aos1917] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
|
16
|
Du H, Han P, Xiang Q, Huang S. MonkeyKing: Adaptive Parameter Tuning on Big Data Platforms with Deep Reinforcement Learning. BIG DATA 2020; 8:270-290. [PMID: 32654536 DOI: 10.1089/big.2019.0123] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Choosing the right parameter configurations for recurring jobs running on big data analytics platforms is difficult because there can be hundreds of possible parameter configurations to pick from. Even the selection of parameter configurations is based on different types of applications and user requirements. The difference between the best configuration and the worst configuration can have a performance impact of more than 10 times. However, parameters of big data platforms are not independent, which makes it a challenge to automatically identify the optimal configuration for a broad spectrum of applications. To alleviate these problems, we proposed MonkeyKing, a system that leverages past experience and collects new information to adjust parameter configurations of big data platforms. It can recommend key parameters, which have strong impact on performance according to job types, and then combine deep reinforcement learning (DRL) to optimize key parameters to improve job performance. We choose the current popular deep Q-network (DQN) structure and its four improved algorithms, including DQN, Double DQN, Dueling DQN, and the combined Double DQN and Dueling DQN, and finally found that the combined Double DQN and Dueling DQN has a better effect. Our experiments and evaluations on Spark show that performance can be improved by ∼25% under best conditions.
Collapse
Affiliation(s)
- Haizhou Du
- College of Electronics and Information Engineering, Tongji University, Shanghai, China
- School of Computer Science and Technology, Shanghai University of Electric Power, Shanghai, China
| | - Ping Han
- School of Computer Science and Technology, Shanghai University of Electric Power, Shanghai, China
| | - Qiao Xiang
- Department of Computer Science, Yale University, New Haven, Connecticut, USA
| | - Sheng Huang
- School of Computer Science and Technology, Shanghai University of Electric Power, Shanghai, China
| |
Collapse
|
17
|
Huang PH. Postselection Inference in Structural Equation Modeling. MULTIVARIATE BEHAVIORAL RESEARCH 2020; 55:344-360. [PMID: 31304796 DOI: 10.1080/00273171.2019.1634996] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
Most statistical inference methods were established under the assumption that the fitted model is known in advance. In practice, however, researchers often obtain their final model by some data-driven selection process. The selection process makes the finally fitted model random, and it also influences the sampling distribution of the estimator. Therefore, implementing naive inference methods may result in wrong conclusions-which is probably a prime source of the reproducibility crisis in psychological science. The present study accommodates three valid state-of-the-art postselection inference methods for structural equation modeling (SEM) from the statistical literature: data splitting (DS), postselection inference (PoSI), and the polyhedral (PH) method. A simulation is conducted to compare the three methods with the commonly used naive procedure under selection events made by L1-penalized SEM. The results show that the naive method often yields incorrect inference, and that the valid methods control the coverage rate in most cases with their own pros and cons. Real world data examples show the practical use of the valid inference methods.
Collapse
Affiliation(s)
- Po-Hsien Huang
- Department of Psychology, National Cheng Kung University
| |
Collapse
|
18
|
Guan L, Tibshirani R. Post model‐fitting exploration via a “Next‐Door” analysis. CAN J STAT 2020; 48:447-470. [DOI: 10.1002/cjs.11542] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Affiliation(s)
- Leying Guan
- Department of Biostatistics Yale University New Haven CT U.S.A
| | - Robert Tibshirani
- Departments of Biomedical Data Sciences, and Statistics Stanford University Stanford CA U.S.A
| |
Collapse
|
19
|
Bachoc F, Preinerstorfer D, Steinberger L. Uniformly valid confidence intervals post-model-selection. Ann Stat 2020. [DOI: 10.1214/19-aos1815] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
|
20
|
Rinaldo A, Wasserman L, G’Sell M. Bootstrapping and sample splitting for high-dimensional, assumption-lean inference. Ann Stat 2019. [DOI: 10.1214/18-aos1784] [Citation(s) in RCA: 20] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
|
21
|
Claeskens G, Cunen C, Hjort NL. Model Selection via Focused Information Criteria for Complex Data in Ecology and Evolution. Front Ecol Evol 2019. [DOI: 10.3389/fevo.2019.00415] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022] Open
|
22
|
Wang W, Sun Y. Penalized local polynomial regression for spatial data. Biometrics 2019; 75:1179-1190. [DOI: 10.1111/biom.13077] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2018] [Accepted: 04/16/2019] [Indexed: 11/29/2022]
Affiliation(s)
- Wu Wang
- Computer, Electrical and Mathematical Sciences and Engineering Division (CEMSE)King Abdullah University of Science and Technology (KAUST)Thuwal Saudi Arabia
| | - Ying Sun
- Computer, Electrical and Mathematical Sciences and Engineering Division (CEMSE)King Abdullah University of Science and Technology (KAUST)Thuwal Saudi Arabia
| |
Collapse
|
23
|
Bachoc F, Leeb H, Pötscher BM. Valid confidence intervals for post-model-selection predictors. Ann Stat 2019. [DOI: 10.1214/18-aos1721] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
|
24
|
Abstract
Summary
The lasso has been studied extensively as a tool for estimating the coefficient vector in the high-dimensional linear model; however, considerably less is known about estimating the error variance in this context. In this paper, we propose the natural lasso estimator for the error variance, which maximizes a penalized likelihood objective. A key aspect of the natural lasso is that the likelihood is expressed in terms of the natural parameterization of the multi-parameter exponential family of a Gaussian with unknown mean and variance. The result is a remarkably simple estimator of the error variance with provably good performance in terms of mean squared error. These theoretical results do not require placing any assumptions on the design matrix or the true regression coefficients. We also propose a companion estimator, called the organic lasso, which theoretically does not require tuning of the regularization parameter. Both estimators do well empirically compared to pre-existing methods, especially in settings where successful recovery of the true support of the coefficient vector is hard. Finally, we show that existing methods can do well under fewer assumptions than previously known, thus providing a fuller story about the problem of estimating the error variance in high-dimensional linear models.
Collapse
Affiliation(s)
- Guo Yu
- Department of Statistics, University of Washington, Box 354322, Seattle, Washington 98105, USA
| | - Jacob Bien
- Data Sciences and Operations, Marshall School of Business, University of Southern California, 3670 Trousdale Pkwy, Los Angeles, California 90089, U.S.A.
| |
Collapse
|
25
|
Pelawa Watagoda LCR, Olive DJ. Bootstrapping multiple linear regression after variable selection. Stat Pap (Berl) 2019. [DOI: 10.1007/s00362-019-01108-9] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
26
|
Tibshirani RJ, Rinaldo A, Tibshirani R, Wasserman L. Uniform asymptotic inference and the bootstrap after model selection. Ann Stat 2018. [DOI: 10.1214/17-aos1584] [Citation(s) in RCA: 36] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
|
27
|
Bachoc F, Blanchard G, Neuvial P. On the post selection inference constant under restricted isometry properties. Electron J Stat 2018. [DOI: 10.1214/18-ejs1490] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
|