1
|
Rosenbaum PR. A second evidence factor for a second control group. Biometrics 2023; 79:3968-3980. [PMID: 37563803 DOI: 10.1111/biom.13921] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2023] [Accepted: 07/24/2023] [Indexed: 08/12/2023]
Abstract
In an observational study of the effects caused by a treatment, a second control group is used in an effort to detect bias from unmeasured covariates, and the investigator is content if no evidence of bias is found. This strategy is not entirely satisfactory: two control groups may differ significantly, yet the difference may be too small to invalidate inferences about the treatment, or the control groups may not differ yet nonetheless fail to provide a tangible strengthening of the evidence of a treatment effect. Is a firmer conclusion possible? Is there a way to analyze a second control group such that the data might report measurably strengthened evidence of cause and effect, that is, insensitivity to larger unmeasured biases? Evidence factor analyses are not commonly used with a second control group: most analyses compare the treated group to each control group, but analyses of that kind are partially redundant; so, they do not constitute evidence factors. An alternative analysis is proposed here, one that does yield two evidence factors, and with a carefully designed test statistic, is capable of extracting strong evidence from the second factor. The new technical work here concerns the development of a test statistic with high design sensitivity and high Bahadur efficiency in a sensitivity analysis for the second factor. A study of binge drinking as a cause of high blood pressure is used as an illustration.
Collapse
Affiliation(s)
- Paul R Rosenbaum
- Department of Statistics and Data Science, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| |
Collapse
|
2
|
Zhang B, Weiss J, Small DS, Zhao Q. Selecting and Ranking Individualized Treatment Rules With Unmeasured Confounding. J Am Stat Assoc 2021. [DOI: 10.1080/01621459.2020.1736083] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
Affiliation(s)
- Bo Zhang
- Department of Statistics, The Wharton School, University of Pennsylvania, Philadelphia, PA
| | - Jordan Weiss
- Department of Sociology, University of Pennsylvania, Philadelphia, PA
| | - Dylan S. Small
- Department of Statistics, The Wharton School, University of Pennsylvania, Philadelphia, PA
| | - Qingyuan Zhao
- Statistical Laboratory, University of Cambridge, Cambridge, UK
| |
Collapse
|
3
|
Karmakar B, Small DS. Assessment of the extent of corroboration of an elaborate theory of a causal hypothesis using partial conjunctions of evidence factors. Ann Stat 2020. [DOI: 10.1214/19-aos1929] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
|
4
|
Abstract
Summary
A sensitivity analysis in an observational study tests whether the qualitative conclusions of an analysis would change if we were to allow for the possibility of limited bias due to confounding. The design sensitivity of a hypothesis test quantifies the asymptotic performance of the test in a sensitivity analysis against a particular alternative. We propose a new, nonasymptotic, distribution-free test, the uniform general signed rank test, for observational studies with paired data, and examine its performance under Rosenbaum’s sensitivity analysis model. Our test can be viewed as adaptively choosing from among a large underlying family of signed rank tests, and we show that the uniform test achieves design sensitivity equal to the maximum design sensitivity over the underlying family of signed rank tests. Our test thus achieves superior design sensitivity, indicating it will perform well in sensitivity analyses on large samples. We support this conclusion with simulations and a data example, showing that the advantages of our test extend to moderate sample sizes as well.
Collapse
Affiliation(s)
- S R Howard
- The Voleon Group, 2150 Dwight Way, Berkeley, California 94704, U.S.A
| | - S D Pimentel
- Department of Statistics, University of California, Berkeley, 367 Evans Hall, Berkeley, California 94720, U.S.A
| |
Collapse
|
5
|
Yu R. Evaluating and improving a matched comparison of antidepressants and bone density. Biometrics 2020; 77:1276-1288. [PMID: 32940344 DOI: 10.1111/biom.13374] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/29/2020] [Revised: 09/04/2020] [Accepted: 09/07/2020] [Indexed: 10/23/2022]
Abstract
Matching is a common approach to covariate adjustment in estimating causal effects in observational studies. It is important to assess covariate balance of the matched samples. This is usually done informally, in ways that have a number of limitations. First, there are many diagnostics, even if covariates are assessed one at a time, which raises multiplicity issues. In addition, joint distributions of covariates, even bivariate distributions, are often ignored. Further, it is an open question whether diagnostics identify the major problems. To address these issues, a formal assessment of covariate balance is developed in the current paper. Unlike the common informal diagnostics, the proposed method compares both marginal distributions and joint distributions of the matched sample with those of the benchmark, complete randomizations. The method controls the probability of falsely identifying a covariate imbalance among many comparisons, yet it has a high probability of correctly detecting and identifying a major problem. An R package met implementing the proposed method is available on CRAN.
Collapse
Affiliation(s)
- Ruoqi Yu
- Department of Statistics, Wharton School, University of Pennsylvania, Philadelphia, Pennsylvania
| |
Collapse
|
6
|
Shauly-Aharonov M. An exact test with high power and robustness to unmeasured confounding effects. Stat Med 2020; 39:1041-1053. [PMID: 31907979 DOI: 10.1002/sim.8460] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2019] [Revised: 12/07/2019] [Accepted: 12/10/2019] [Indexed: 11/11/2022]
Abstract
In observational studies, it is agreed that the sensitivity of the findings to unmeasured confounders needs to be assessed. The issue is that a poor choice of test statistic can result in overstated sensitivity to hidden bias of this kind. In this article, a new adaptive test is proposed, guided by considerations of low sensitivity to hidden bias: it is tailored so that its power is greater than other leading tests, both in finite and infinite samples. One way of defining power in case of possible confounders is as the probability of reporting robustness (ie, insensitivity) of a true discovery to potential bias. In case of finite samples, we compute the power by simulations. When sample size approaches infinity, a meaningful indicator of the power is the design sensitivity, which is computed analytically and found to be better in the new test than in existing tests. Another asymptotic criterion for comparing tests when there is concern for confounders is Bahadur efficiency. The proposed test outperforms commonly used tests in terms of Bahadur efficiency in most sampling situations. The advantages of the new test mainly stem from its adaptivity: it combines two test statistics and consequently achieves the best design sensitivity and the best Bahadur efficiency of the two. As a "real-world" examination, we compare 441 daily smokers to 441 nonsmokers, to test the effect of smoking on periodontal disease. The new test is more robust to unmeasured confounders than both the Wilcoxon signed rank test and the paired t-test.
Collapse
|
7
|
Karmakar B, French B, Small DS. Integrating the evidence from evidence factors in observational studies. Biometrika 2019. [DOI: 10.1093/biomet/asz003] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022] Open
Abstract
Summary
A sensitivity analysis for an observational study assesses how much bias, due to nonrandom assignment of treatment, would be necessary to change the conclusions of an analysis that assumes treatment assignment was effectively random. The evidence for a treatment effect can be strengthened if two different analyses, which could be affected by different types of biases, are both somewhat insensitive to bias. The finding from the observational study is then said to be replicated. Evidence factors allow for two independent analyses to be constructed from the same dataset. When combining the evidence factors, the Type I error rate must be controlled to obtain valid inference. A powerful method is developed for controlling the familywise error rate for sensitivity analyses with evidence factors. It is shown that the Bahadur efficiency of sensitivity analysis for the combined evidence is greater than for either evidence factor alone. The proposed methods are illustrated through a study of the effect of radiation exposure on the risk of cancer. An R package, evidenceFactors, is available from CRAN to implement the methods of the paper.
Collapse
Affiliation(s)
- B Karmakar
- Department of Statistics, The Wharton School, University of Pennsylvania, 3730 Walnut Street, Philadelphia, Pennsylvania 19104-6340, U.S.A
| | - B French
- Department of Statistics, Radiation Effects Research Foundation, 5-2 Hijiyama Park, Minami-ku, Hiroshima 732-0815, Japan
| | - D S Small
- Department of Statistics, The Wharton School, University of Pennsylvania, 3730 Walnut Street, Philadelphia, Pennsylvania 19104-6340, U.S.A
| |
Collapse
|
8
|
Affiliation(s)
- Qingyuan Zhao
- Department of Statistics, The Wharton School, University of Pennsylvania, Philadelphia, PA
| |
Collapse
|
9
|
Zhao Q, Small DS, Rosenbaum PR. Cross-Screening in Observational Studies That Test Many Hypotheses. J Am Stat Assoc 2018. [DOI: 10.1080/01621459.2017.1407770] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
Affiliation(s)
- Qingyuan Zhao
- Department of Statistics, The Wharton School, University of Pennsylvania, Philadelphia, PA
| | - Dylan S. Small
- Department of Statistics, The Wharton School, University of Pennsylvania, Philadelphia, PA
| | - Paul R. Rosenbaum
- Department of Statistics, The Wharton School, University of Pennsylvania, Philadelphia, PA
| |
Collapse
|
10
|
Ertefaie A, Small DS, Rosenbaum PR. Quantitative Evaluation of the Trade-Off of Strengthened Instruments and Sample Size in Observational Studies. J Am Stat Assoc 2018. [DOI: 10.1080/01621459.2017.1305275] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Affiliation(s)
- Ashkan Ertefaie
- Department of Biostatistics and Computational Biology at the University of Rochester, Rochester, NY
| | - Dylan S. Small
- Department of Statistics, Wharton School, University of Pennsylvania, Philadelphia, PA
| | - Paul R. Rosenbaum
- Department of Statistics, Wharton School, University of Pennsylvania, Philadelphia, PA
| |
Collapse
|
11
|
Lee K, Small DS, Rosenbaum PR. A powerful approach to the study of moderate effect modification in observational studies. Biometrics 2018; 74:1161-1170. [PMID: 29738603 DOI: 10.1111/biom.12884] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2017] [Revised: 03/01/2018] [Accepted: 03/01/2018] [Indexed: 11/28/2022]
Abstract
Effect modification means the magnitude or stability of a treatment effect varies as a function of an observed covariate. Generally, larger and more stable treatment effects are insensitive to larger biases from unmeasured covariates, so a causal conclusion may be considerably firmer if this pattern is noted if it occurs. We propose a new strategy, called the submax-method, that combines exploratory, and confirmatory efforts to determine whether there is stronger evidence of causality-that is, greater insensitivity to unmeasured confounding-in some subgroups of individuals. It uses the joint distribution of test statistics that split the data in various ways based on certain observed covariates. For L binary covariates, the method splits the population L times into two subpopulations, perhaps first men and women, perhaps then smokers and nonsmokers, computing a test statistic from each subpopulation, and appends the test statistic for the whole population, making <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mn>2</mml:mn> <mml:mi>L</mml:mi> <mml:mo>+</mml:mo> <mml:mn>1</mml:mn></mml:math> test statistics in total. Although L binary covariates define <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:msup><mml:mn>2</mml:mn> <mml:mi>L</mml:mi></mml:msup> </mml:math> interaction groups, only <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mn>2</mml:mn> <mml:mi>L</mml:mi> <mml:mo>+</mml:mo> <mml:mn>1</mml:mn></mml:math> tests are performed, and at least <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>L</mml:mi> <mml:mo>+</mml:mo> <mml:mn>1</mml:mn></mml:math> of these tests use at least half of the data. The submax-method achieves the highest design sensitivity and the highest Bahadur efficiency of its component tests. Moreover, the form of the test is sufficiently tractable that its large sample power may be studied analytically. The simulation suggests that the submax method exhibits superior performance, in comparison with an approach using CART, when there is effect modification of moderate size. Using data from the NHANES I epidemiologic follow-up survey, an observational study of the effects of physical activity on survival is used to illustrate the method. The method is implemented in the <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>R</mml:mi></mml:math> package <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>submax</mml:mi></mml:math> which contains the NHANES example. An online Appendix provides simulation results and further analysis of the example.
Collapse
Affiliation(s)
- Kwonsang Lee
- Department of Biostatistics, Harvard School of Public Health, Boston, Massachusetts 02115, U.S.A
| | - Dylan S Small
- Department of Statistics, University of Pennsylvania, Philadelphia, Pennsylvania 19104, U.S.A
| | - Paul R Rosenbaum
- Department of Statistics, University of Pennsylvania, Philadelphia, Pennsylvania 19104, U.S.A
| |
Collapse
|
12
|
Rosenbaum PR. Using Scheffé projections for multiple outcomes in an observational study of smoking and periodontal disease. Ann Appl Stat 2016. [DOI: 10.1214/16-aoas942] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
|
13
|
Dai H, Wu G, Wu M, Zhi D. An Optimal Bahadur-Efficient Method in Detection of Sparse Signals with Applications to Pathway Analysis in Sequencing Association Studies. PLoS One 2016; 11:e0152667. [PMID: 27380176 PMCID: PMC4933358 DOI: 10.1371/journal.pone.0152667] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/12/2015] [Accepted: 03/17/2016] [Indexed: 11/18/2022] Open
Abstract
Next-generation sequencing data pose a severe curse of dimensionality, complicating traditional "single marker-single trait" analysis. We propose a two-stage combined p-value method for pathway analysis. The first stage is at the gene level, where we integrate effects within a gene using the Sequence Kernel Association Test (SKAT). The second stage is at the pathway level, where we perform a correlated Lancaster procedure to detect joint effects from multiple genes within a pathway. We show that the Lancaster procedure is optimal in Bahadur efficiency among all combined p-value methods. The Bahadur efficiency,[Formula: see text], compares sample sizes among different statistical tests when signals become sparse in sequencing data, i.e. ε →0. The optimal Bahadur efficiency ensures that the Lancaster procedure asymptotically requires a minimal sample size to detect sparse signals ([Formula: see text]). The Lancaster procedure can also be applied to meta-analysis. Extensive empirical assessments of exome sequencing data show that the proposed method outperforms Gene Set Enrichment Analysis (GSEA). We applied the competitive Lancaster procedure to meta-analysis data generated by the Global Lipids Genetics Consortium to identify pathways significantly associated with high-density lipoprotein cholesterol, low-density lipoprotein cholesterol, triglycerides, and total cholesterol.
Collapse
Affiliation(s)
- Hongying Dai
- Health Services and Outcomes Research, Children’s Mercy Hospital, Kansas City, MO, United States of America
- Department of Biomedical & Health Informatics, University of Missouri-Kansas City, Kansas City, MO, United States of America
- * E-mail:
| | - Guodong Wu
- Lovelace Respiratory Research Institute, Albuquerque, New Mexico, United States of America
| | - Michael Wu
- Biostatistics and Biomathematics Program, Public Health Sciences Division, Fred Hutchinson Cancer Research Center, Seattle, WA, United States of America
| | - Degui Zhi
- Department of Biostatistics, University of Alabama at Birmingham, Birmingham, AL, United States of America
| |
Collapse
|
14
|
|