1
|
Rosenbaum PR. A second evidence factor for a second control group. Biometrics 2023; 79:3968-3980. [PMID: 37563803 DOI: 10.1111/biom.13921] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2023] [Accepted: 07/24/2023] [Indexed: 08/12/2023]
Abstract
In an observational study of the effects caused by a treatment, a second control group is used in an effort to detect bias from unmeasured covariates, and the investigator is content if no evidence of bias is found. This strategy is not entirely satisfactory: two control groups may differ significantly, yet the difference may be too small to invalidate inferences about the treatment, or the control groups may not differ yet nonetheless fail to provide a tangible strengthening of the evidence of a treatment effect. Is a firmer conclusion possible? Is there a way to analyze a second control group such that the data might report measurably strengthened evidence of cause and effect, that is, insensitivity to larger unmeasured biases? Evidence factor analyses are not commonly used with a second control group: most analyses compare the treated group to each control group, but analyses of that kind are partially redundant; so, they do not constitute evidence factors. An alternative analysis is proposed here, one that does yield two evidence factors, and with a carefully designed test statistic, is capable of extracting strong evidence from the second factor. The new technical work here concerns the development of a test statistic with high design sensitivity and high Bahadur efficiency in a sensitivity analysis for the second factor. A study of binge drinking as a cause of high blood pressure is used as an illustration.
Collapse
Affiliation(s)
- Paul R Rosenbaum
- Department of Statistics and Data Science, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| |
Collapse
|
2
|
Rosenbaum PR. Sensitivity analyses informed by tests for bias in observational studies. Biometrics 2023; 79:475-487. [PMID: 34505285 DOI: 10.1111/biom.13558] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2021] [Revised: 08/01/2021] [Accepted: 08/19/2021] [Indexed: 11/29/2022]
Abstract
In an observational study, the treatment received and the outcome exhibited may be associated in the absence of an effect caused by the treatment, even after controlling for observed covariates. Two tactics are common: (i) a test for unmeasured bias may be obtained using a secondary outcome for which the effect is known and (ii) a sensitivity analysis may explore the magnitude of unmeasured bias that would need to be present to explain the observed association as something other than an effect caused by the treatment. Can such a test for unmeasured bias inform the sensitivity analysis? If the test for bias does not discover evidence of unmeasured bias, then ask: Are conclusions therefore insensitive to larger unmeasured biases? Conversely, if the test for bias does find evidence of bias, then ask: What does that imply about sensitivity to biases? This problem is formulated in a new way as a convex quadratically constrained quadratic program and solved on a large scale using interior point methods by a modern solver. That is, a convex quadratic function of N variables is minimized subject to constraints on linear and convex quadratic functions of these variables. The quadratic function that is minimized is a statistic for the primary outcome that is a function of the unknown treatment assignment probabilities. The quadratic function that constrains this minimization is a statistic for subsidiary outcome that is also a function of these same unknown treatment assignment probabilities. In effect, the first statistic is minimized over a confidence set for the unknown treatment assignment probabilities supplied by the unaffected outcome. This process avoids the mistake of interpreting the failure to reject a hypothesis as support for the truth of that hypothesis. The method is illustrated by a study of the effects of light daily alcohol consumption on high-density lipoprotein (HDL) cholesterol levels. In this study, the method quickly optimizes a nonlinear function of N = 800 $N=800$ variables subject to linear and quadratic constraints. In the example, strong evidence of unmeasured bias is found using the subsidiary outcome, but, perhaps surprisingly, this finding makes the primary comparison insensitive to larger biases.
Collapse
Affiliation(s)
- Paul R Rosenbaum
- Department of Statistics and Data Science, University of Pennsylvania, Philadelphia, Pennsylvania
| |
Collapse
|
3
|
Chen H, Heitjan DF. Analysis of local sensitivity to nonignorability with missing outcomes and predictors. Biometrics 2022; 78:1342-1352. [PMID: 34297356 DOI: 10.1111/biom.13532] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2020] [Revised: 07/03/2021] [Accepted: 07/15/2021] [Indexed: 12/30/2022]
Abstract
The ISNI (index of sensitivity to local nonignorability) method quantifies local sensitivity of parametric inferences to nonignorable missingness in an outcome variable. Here we extend ISNI to the situations where both outcomes and predictors can be missing and where the missingness mechanism can be either parametric or semi-parametric. We define the quantity MinNI (minimum nonignorability) to be an approximation to the norm of the smallest value of the transformed nonignorability that gives a nonnegligible displacement of the estimate of the parameter of interest. We illustrate our method in a complete data set from which we synthetically delete observations according to various patterns. We then apply the method to real-data examples involving the normal linear model and conditional logistic regression.
Collapse
Affiliation(s)
- Heng Chen
- Public Health Sciences Division, Fred Hutchinson Cancer Research Center, Seattle, Washington, USA
| | - Daniel F Heitjan
- Department of Statistical Science, Southern Methodist University, Dallas, Texas, USA.,Department of Population & Data Sciences, University of Texas Southwestern Medical Center, Dallas, Texas, USA
| |
Collapse
|
4
|
Zhang B, Tchetgen Tchetgen EJ. A Semiparametric Approach to Model-Based Sensitivity Analysis in Observational Studies. JOURNAL OF THE ROYAL STATISTICAL SOCIETY. SERIES A, (STATISTICS IN SOCIETY) 2022; 185:S668-S691. [PMID: 36777968 PMCID: PMC9916707 DOI: 10.1111/rssa.12946] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/18/2023]
Abstract
When drawing causal inference from observational data, there is almost always concern about unmeasured confounding. One way to tackle this is to conduct a sensitivity analysis. One widely-used sensitivity analysis framework hypothesizes the existence of a scalar unmeasured confounder U and asks how the causal conclusion would change were U measured and included in the primary analysis. Work along this line often makes various parametric assumptions on U, for the sake of mathematical and computational convenience. In this article, we further this line of research by developing a valid sensitivity analysis that leaves the distribution of U unrestricted. Compared to many existing methods in the literature, our method allows for a larger and more flexible family of models, mitigates observable implications (Franks et al., 2019), and works seamlessly with any primary analysis that models the outcome regression parametrically. We construct both pointwise confidence intervals and confidence bands that are uniformly valid over a given sensitivity parameter space, thus formally accounting for unknown sensitivity parameters. We apply our proposed method on an influential yet controversial study of the causal relationship between war experiences and political activeness using observational data from Uganda.
Collapse
Affiliation(s)
- Bo Zhang
- Vaccine and Infectious Disease Division, Fred Hutchinson Cancer Center, Seattle, WA, U.S.A
| | - Eric J Tchetgen Tchetgen
- Department of Statistics and Data Science, The Wharton School, University of Pennsylvania, Philadelphia, PA, U.S.A
| |
Collapse
|
5
|
Chen K, Heng S, Long Q, Zhang B. Testing Biased Randomization Assumptions and Quantifying Imperfect Matching and Residual Confounding in Matched Observational Studies. J Comput Graph Stat 2022; 32:528-538. [PMID: 37334200 PMCID: PMC10275332 DOI: 10.1080/10618600.2022.2116447] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2021] [Accepted: 08/17/2022] [Indexed: 10/24/2022]
Abstract
One central goal of design of observational studies is to embed non-experimental data into an approximate randomized controlled trial using statistical matching. Despite empirical researchers' best intention and effort to create high-quality matched samples, residual imbalance due to observed covariates not being well matched often persists. Although statistical tests have been developed to test the randomization assumption and its implications, few provide a means to quantify the level of residual confounding due to observed covariates not being well matched in matched samples. In this article, we develop two generic classes of exact statistical tests for a biased randomization assumption. One important by-product of our testing framework is a quantity called residual sensitivity value (RSV), which provides a means to quantify the level of residual confounding due to imperfect matching of observed covariates in a matched sample. We advocate taking into account RSV in the downstream primary analysis. The proposed methodology is illustrated by re-examining a famous observational study concerning the effect of right heart catheterization (RHC) in the initial care of critically ill patients. Code implementing the method can be found in the supplementary materials.
Collapse
Affiliation(s)
- Kan Chen
- Graduate Group of Applied Mathematics and Computational Science, School of Arts and Sciences, University of Pennsylvania, Philadelphia, Pennsylvania, U.S.A
| | - Siyu Heng
- Department of Biostatistics, School of Global Public Health, New York University, New York City, New York, U.S.A
| | - Qi Long
- Department of Biostatistics, Epidemiology and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, U.S.A
| | - Bo Zhang
- Vaccine and Infectious Disease Division, Fred Hutchinson Cancer Center, Seattle, Washington, U.S.A
| |
Collapse
|
6
|
Rosenbaum PR. A statistic with demonstrated insensitivity to unmeasured bias for 2 × 2 × S tables in observational studies. Stat Med 2022; 41:3758-3771. [PMID: 35607846 DOI: 10.1002/sim.9446] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2021] [Revised: 04/08/2022] [Accepted: 05/10/2022] [Indexed: 11/10/2022]
Abstract
Are weak associations between a treatment and a binary outcome always sensitive to small unmeasured biases in observational studies? This possibility is often discussed in epidemiology. The familiar Mantel-Haenszel test for a 2 × 2 × S $$ 2\times 2\times S $$ contingency table exaggerates sensitivity to unmeasured biases when the population odds ratios vary among the S $$ S $$ strata. A statistic built from several components, here from the S $$ S $$ strata, is said to have demonstrated insensitivity to bias if it uses only those components that provide indications of insensitivity to bias. Briefly, such a statistic is a d $$ d $$ -statistic. There are 2 S - 1 $$ {2}^S-1 $$ candidate statistics with S $$ S $$ strata, and a d $$ d $$ -statistic considers them all. To have level α $$ \alpha $$ , a test based on a d $$ d $$ -statistic must pay a price for its double use of the data, but as the sample size increases, that price becomes small, while the gain may be large. The price is paid by conditioning on the limited information used to identify components that are insensitive to a bias of specified magnitude, basing the test result on the information that remains after conditioning. In large samples, the d $$ d $$ -statistic achieves the largest possible design sensitivity, so it does not exaggerate sensitivity to unmeasured bias. A simulation verifies that the large sample result has traction in samples of practical size. A study of sunlight as a cause of cataract is used to illustrate issues and methods. Several extensions of the method are discussed. An R package dstat2x2xk implements the method.
Collapse
Affiliation(s)
- Paul R Rosenbaum
- Department of Statistics and Data Science, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| |
Collapse
|
7
|
Fogarty CB, Lee K, Kelz RR, Keele LJ. Biased Encouragements and Heterogeneous Effects in an Instrumental Variable Study of Emergency General Surgical Outcomes. J Am Stat Assoc 2021. [DOI: 10.1080/01621459.2020.1863220] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
Affiliation(s)
- Colin B. Fogarty
- Operations Research and Statistics Group, MIT Sloan School of Management, Massachusetts Institute of Technology, Cambridge, MA
| | - Kwonsang Lee
- Department of Statistics, Sungkyunkwan University, Seoul, Republic of Korea
| | - Rachel R. Kelz
- Center for Surgery and Health Economics, Department of Surgery, Hospital of the University of Pennsylvania, Philadelphia, PA
| | - Luke J. Keele
- Department of Surgery, University of Pennsylvania, Philadelphia, PA
| |
Collapse
|
8
|
Nattino G, Lu B, Shi J, Lemeshow S, Xiang H. Triplet Matching for Estimating Causal Effects With Three Treatment Arms: A Comparative Study of Mortality by Trauma Center Level. J Am Stat Assoc 2021. [DOI: 10.1080/01621459.2020.1737078] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
Affiliation(s)
- Giovanni Nattino
- Division of Biostatistics, College of Public Health, The Ohio State University, Columbus, OH
| | - Bo Lu
- Division of Biostatistics, College of Public Health, The Ohio State University, Columbus, OH
| | - Junxin Shi
- Center for Pediatric Trauma Research, Center for Injury Research and Policy, Abigail Wexner Research Institute at Nationwide Children’s Hospital, Columbus, OH
| | - Stanley Lemeshow
- Division of Biostatistics, College of Public Health, The Ohio State University, Columbus, OH
| | - Henry Xiang
- Center for Pediatric Trauma Research, Center for Injury Research and Policy, Abigail Wexner Research Institute at Nationwide Children’s Hospital, Columbus, OH
- College of Medicine, The Ohio State University, Columbus, OH
| |
Collapse
|
9
|
Karmakar B, Small DS. Assessment of the extent of corroboration of an elaborate theory of a causal hypothesis using partial conjunctions of evidence factors. Ann Stat 2020. [DOI: 10.1214/19-aos1929] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
|
10
|
Rosenbaum PR. A conditional test with demonstrated insensitivity to unmeasured bias in matched observational studies. Biometrika 2020. [DOI: 10.1093/biomet/asaa032] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022] Open
Abstract
Summary
In an observational study matched for observed covariates, an association between treatment received and outcome exhibited may indicate not an effect caused by the treatment, but merely some bias in the allocation of treatments to individuals within matched pairs. The evidence that distinguishes moderate biases from causal effects is unevenly dispersed among possible comparisons in an observational study: some comparisons are insensitive to larger biases than others. Intuitively, larger treatment effects tend to be insensitive to larger unmeasured biases, and perhaps matched pairs can be grouped using covariates, doses or response patterns so that groups of pairs with larger treatment effects may be identified. Even if an investigator has a reasoned conjecture about where to look for insensitive comparisons, that conjecture might prove mistaken, or, when not mistaken, it might be received sceptically by other scientists who doubt the conjecture or judge it to be too convenient in light of its success with the data at hand. In this article a test is proposed that searches for insensitive findings over many comparisons, but controls the probability of falsely rejecting a true null hypothesis of no treatment effect in the presence of a bias of specified magnitude. An example is studied in which the test considers many comparisons and locates an interpretable comparison that is insensitive to larger biases than a conventional comparison based on Wilcoxon’s signed rank statistic applied to all pairs. A simulation examines the power of the proposed test. The method is implemented in the R package dstat, which contains the example and reproduces the analysis.
Collapse
Affiliation(s)
- P R Rosenbaum
- Department of Statistics, The Wharton School, University of Pennsylvania, Philadelphia, Pennsylvania 19104, U.S.A
| |
Collapse
|
11
|
Shauly-Aharonov M. An exact test with high power and robustness to unmeasured confounding effects. Stat Med 2020; 39:1041-1053. [PMID: 31907979 DOI: 10.1002/sim.8460] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2019] [Revised: 12/07/2019] [Accepted: 12/10/2019] [Indexed: 11/11/2022]
Abstract
In observational studies, it is agreed that the sensitivity of the findings to unmeasured confounders needs to be assessed. The issue is that a poor choice of test statistic can result in overstated sensitivity to hidden bias of this kind. In this article, a new adaptive test is proposed, guided by considerations of low sensitivity to hidden bias: it is tailored so that its power is greater than other leading tests, both in finite and infinite samples. One way of defining power in case of possible confounders is as the probability of reporting robustness (ie, insensitivity) of a true discovery to potential bias. In case of finite samples, we compute the power by simulations. When sample size approaches infinity, a meaningful indicator of the power is the design sensitivity, which is computed analytically and found to be better in the new test than in existing tests. Another asymptotic criterion for comparing tests when there is concern for confounders is Bahadur efficiency. The proposed test outperforms commonly used tests in terms of Bahadur efficiency in most sampling situations. The advantages of the new test mainly stem from its adaptivity: it combines two test statistics and consequently achieves the best design sensitivity and the best Bahadur efficiency of the two. As a "real-world" examination, we compare 441 daily smokers to 441 nonsmokers, to test the effect of smoking on periodontal disease. The new test is more robust to unmeasured confounders than both the Wilcoxon signed rank test and the paired t-test.
Collapse
|
12
|
Fogarty CB. Studentized Sensitivity Analysis for the Sample Average Treatment Effect in Paired Observational Studies. J Am Stat Assoc 2019. [DOI: 10.1080/01621459.2019.1632072] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
Affiliation(s)
- Colin B. Fogarty
- Operations Research and Statistics Group, MIT Sloan School of Management, Massachusetts Institute of Technology, Cambridge, MA
| |
Collapse
|
13
|
Nattino G, Lu B. Model assisted sensitivity analyses for hidden bias with binary outcomes. Biometrics 2018; 74:1141-1149. [PMID: 29992547 DOI: 10.1111/biom.12919] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2017] [Revised: 05/01/2018] [Accepted: 05/01/2018] [Indexed: 11/29/2022]
Abstract
In medical and health sciences, observational studies are a major data source for inferring causal relationships. Unlike randomized experiments, observational studies are vulnerable to the hidden bias introduced by unmeasured confounders. The impact of unmeasured covariates on the causal effect can be assessed by conducting a sensitivity analysis. A comprehensive framework of sensitivity analyses has been developed for matching designs. Sensitivity parameters are introduced to capture the association between the missing covariates and the exposure or the outcome. Fixing sensitivity parameter values, it is possible to compute the bounds of the p-value of a randomization test on causal effects. We propose a model assisted sensitivity analysis with binary outcomes for the general 1:k matching design, which provides results equivalent to the conventional nonparametric approach in large sample. By introducing a conditional logistic outcome model, we substantially simplify the implementation and interpretation of the sensitivity analysis. More importantly, we are able to provide a closed form representation for the set of sensitivity parameters for which the maximum p-values are non-significant. This methodology can be easily extended to matching designs with multilevel treatments. We illustrate our method using a U.S. trauma care database to examine mortality difference between trauma care levels.
Collapse
Affiliation(s)
- Giovanni Nattino
- Division of Biostatistics, College of Public Health, The Ohio State University, Columbus, Ohio, U.S.A
| | - Bo Lu
- Division of Biostatistics, College of Public Health, The Ohio State University, Columbus, Ohio, U.S.A
| |
Collapse
|