1
|
Rosenbaum PR. A second evidence factor for a second control group. Biometrics 2023; 79:3968-3980. [PMID: 37563803 DOI: 10.1111/biom.13921] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2023] [Accepted: 07/24/2023] [Indexed: 08/12/2023]
Abstract
In an observational study of the effects caused by a treatment, a second control group is used in an effort to detect bias from unmeasured covariates, and the investigator is content if no evidence of bias is found. This strategy is not entirely satisfactory: two control groups may differ significantly, yet the difference may be too small to invalidate inferences about the treatment, or the control groups may not differ yet nonetheless fail to provide a tangible strengthening of the evidence of a treatment effect. Is a firmer conclusion possible? Is there a way to analyze a second control group such that the data might report measurably strengthened evidence of cause and effect, that is, insensitivity to larger unmeasured biases? Evidence factor analyses are not commonly used with a second control group: most analyses compare the treated group to each control group, but analyses of that kind are partially redundant; so, they do not constitute evidence factors. An alternative analysis is proposed here, one that does yield two evidence factors, and with a carefully designed test statistic, is capable of extracting strong evidence from the second factor. The new technical work here concerns the development of a test statistic with high design sensitivity and high Bahadur efficiency in a sensitivity analysis for the second factor. A study of binge drinking as a cause of high blood pressure is used as an illustration.
Collapse
Affiliation(s)
- Paul R Rosenbaum
- Department of Statistics and Data Science, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| |
Collapse
|
2
|
Hasegawa RB, Small DS. Estimating Malaria Vaccine Efficacy in the Absence of a Gold Standard Case Definition: Mendelian Factorial Design. J Am Stat Assoc 2022. [DOI: 10.1080/01621459.2020.1863222] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
Affiliation(s)
- Raiden B. Hasegawa
- Department of Statistics, The Wharton School, University of Pennsylvania, Philadelphia, PA
| | - Dylan S. Small
- Department of Statistics, The Wharton School, University of Pennsylvania, Philadelphia, PA
| |
Collapse
|
3
|
Zhao A, Lee Y, Small DS, Karmakar B. Evidence factors from multiple, possibly invalid, instrumental variables. Ann Stat 2022. [DOI: 10.1214/21-aos2148] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
Affiliation(s)
- Anqi Zhao
- Department of Statistics and Data Science, National University of Singapore
| | - Youjin Lee
- Department of Biostatistics, Brown University
| | - Dylan S. Small
- Department of Statistics and Data Science, University of Pennsylvania
| | | |
Collapse
|
4
|
Karmakar B, Small DS, Rosenbaum PR. Reinforced Designs: Multiple Instruments Plus Control Groups as Evidence Factors in an Observational Study of the Effectiveness of Catholic Schools. J Am Stat Assoc 2021. [DOI: 10.1080/01621459.2020.1745811] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
Affiliation(s)
- Bikram Karmakar
- Department of Statistics, University of Florida, Gainesville, FL
| | - Dylan S. Small
- Statistics Department, University of Pennsylvania, Philadelphia, PA
| | | |
Collapse
|
5
|
Karmakar B, Small DS. Assessment of the extent of corroboration of an elaborate theory of a causal hypothesis using partial conjunctions of evidence factors. Ann Stat 2020. [DOI: 10.1214/19-aos1929] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
|
6
|
Karmakar B, Doubeni CA, Small DS. EVIDENCE FACTORS IN A CASE-CONTROL STUDY WITH APPLICATION TO THE EFFECT OF FLEXIBLE SIGMOIDOSCOPY SCREENING ON COLORECTAL CANCER. Ann Appl Stat 2020; 14:829-849. [PMID: 38465229 PMCID: PMC10924422 DOI: 10.1214/20-aoas1329] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/12/2024]
Abstract
As in any observational study, in a case-control study a primary concern is potential unmeasured confounders. Bias, due to unmeasured confounders, can result in a false discovery of an apparent treatment effect when there is none. Replication of an observational study, which tries to provide multiple analyses of the data where the biases affecting each analysis are thought to be different, is one way to strengthen the evidence from an observational study. Evidence factors allow for internal replication by testing a hypothesis using multiple comparisons in a way that the comparisons yield independent evidence and differ in the sources of potential bias. We construct evidence factors in a case-control study in which there are two types of cases, "narrow" cases which are thought to be potentially more affected by the exposure and "marginal" cases which are thought to have more heterogeneous causes. We develop and study an inference procedure for using such evidence factors and apply it to a study of the effect of sigmoidoscopy screening on colorectal cancer.
Collapse
Affiliation(s)
- Bikram Karmakar
- Department of Statistics, College of Liberal Arts and Sciences, University of Florida
| | - Chyke A Doubeni
- Center for Health Equity and Community Engagement Research, Mayo Clinic
| | - Dylan S Small
- Department of Statistics, The Wharton School, University of Pennsylvania
| |
Collapse
|
7
|
Karmakar B, Small DS, Rosenbaum PR. Using Evidence Factors to Clarify Exposure Biomarkers. Am J Epidemiol 2020; 189:243-249. [PMID: 31912138 DOI: 10.1093/aje/kwz263] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2019] [Revised: 08/15/2019] [Accepted: 08/26/2019] [Indexed: 11/14/2022] Open
Abstract
A study has 2 evidence factors if it permits 2 statistically independent inferences about 1 treatment effect such that each factor is immune to some bias that would invalidate the other factor. Because the 2 factors are statistically independent, the evidence they provide can be combined using methods associated with meta-analysis for independent studies, despite using the same data twice in different ways. We illustrate evidence factors, applying them in a new way in investigations that have both an exposure biomarker and a coarse external measure of exposure to a treatment. To illustrate, we consider the possible effects of cigarette smoking on homocysteine levels, with self-reported smoking and a cotinine biomarker. We examine joint sensitivity of 2 factors to bias from confounding, a central aspect of any observational study.
Collapse
Affiliation(s)
- Bikram Karmakar
- Department of Statistics, College of Liberal Arts and Sciences, University of Florida, Gainesville, Florida
| | - Dylan S Small
- Department of Statistics, the Wharton School, University of Pennsylvania, Philadelphia, Pennsylvania
| | - Paul R Rosenbaum
- Department of Statistics, the Wharton School, University of Pennsylvania, Philadelphia, Pennsylvania
| |
Collapse
|
8
|
Karmakar B, Small DS, Rosenbaum PR. Using Approximation Algorithms to Build Evidence Factors and Related Designs for Observational Studies. J Comput Graph Stat 2019. [DOI: 10.1080/10618600.2019.1584900] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
Affiliation(s)
- Bikram Karmakar
- Wharton School, Department of Statistics, University of Pennsylvania, Philadelphia, PA
| | - Dylan S. Small
- Wharton School, Department of Statistics, University of Pennsylvania, Philadelphia, PA
| | - Paul R. Rosenbaum
- Wharton School, Department of Statistics, University of Pennsylvania, Philadelphia, PA
| |
Collapse
|
9
|
Karmakar B, French B, Small DS. Integrating the evidence from evidence factors in observational studies. Biometrika 2019. [DOI: 10.1093/biomet/asz003] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022] Open
Abstract
Summary
A sensitivity analysis for an observational study assesses how much bias, due to nonrandom assignment of treatment, would be necessary to change the conclusions of an analysis that assumes treatment assignment was effectively random. The evidence for a treatment effect can be strengthened if two different analyses, which could be affected by different types of biases, are both somewhat insensitive to bias. The finding from the observational study is then said to be replicated. Evidence factors allow for two independent analyses to be constructed from the same dataset. When combining the evidence factors, the Type I error rate must be controlled to obtain valid inference. A powerful method is developed for controlling the familywise error rate for sensitivity analyses with evidence factors. It is shown that the Bahadur efficiency of sensitivity analysis for the combined evidence is greater than for either evidence factor alone. The proposed methods are illustrated through a study of the effect of radiation exposure on the risk of cancer. An R package, evidenceFactors, is available from CRAN to implement the methods of the paper.
Collapse
Affiliation(s)
- B Karmakar
- Department of Statistics, The Wharton School, University of Pennsylvania, 3730 Walnut Street, Philadelphia, Pennsylvania 19104-6340, U.S.A
| | - B French
- Department of Statistics, Radiation Effects Research Foundation, 5-2 Hijiyama Park, Minami-ku, Hiroshima 732-0815, Japan
| | - D S Small
- Department of Statistics, The Wharton School, University of Pennsylvania, 3730 Walnut Street, Philadelphia, Pennsylvania 19104-6340, U.S.A
| |
Collapse
|
10
|
French B, Cologne J, Sakata R, Utada M, Preston DL. Selection of reference groups in the Life Span Study of atomic bomb survivors. Eur J Epidemiol 2017; 32:1055-1063. [DOI: 10.1007/s10654-017-0337-9] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2017] [Accepted: 11/23/2017] [Indexed: 11/24/2022]
|
11
|
|
12
|
Richardson A, Hudgens MG, Gilbert PB, Fine JP. Nonparametric Bounds and Sensitivity Analysis of Treatment Effects. Stat Sci 2014; 29:596-618. [PMID: 25663743 PMCID: PMC4317325 DOI: 10.1214/14-sts499] [Citation(s) in RCA: 24] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
Abstract
This paper considers conducting inference about the effect of a treatment (or exposure) on an outcome of interest. In the ideal setting where treatment is assigned randomly, under certain assumptions the treatment effect is identifiable from the observable data and inference is straightforward. However, in other settings such as observational studies or randomized trials with noncompliance, the treatment effect is no longer identifiable without relying on untestable assumptions. Nonetheless, the observable data often do provide some information about the effect of treatment, that is, the parameter of interest is partially identifiable. Two approaches are often employed in this setting: (i) bounds are derived for the treatment effect under minimal assumptions, or (ii) additional untestable assumptions are invoked that render the treatment effect identifiable and then sensitivity analysis is conducted to assess how inference about the treatment effect changes as the untestable assumptions are varied. Approaches (i) and (ii) are considered in various settings, including assessing principal strata effects, direct and indirect effects and effects of time-varying exposures. Methods for drawing formal inference about partially identified parameters are also discussed.
Collapse
Affiliation(s)
- Amy Richardson
- Quantitative Analyst, Google Inc., Mountain View, California 94043, USA
| | - Michael G. Hudgens
- Associate Professor, Department of Biostatistics, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina 27599, USA
| | - Peter B. Gilbert
- Member, Statistical Center for HIV/AIDS Research and Prevention (SCHARP), Fred Hutchinson Cancer Research Center, Seattle, Washington 98109-1024, USA
| | - Jason P. Fine
- Professor, Department of Biostatistics, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina 27599, USA
| |
Collapse
|
13
|
Rosenbaum PR. Weighted M-statistics With Superior Design Sensitivity in Matched Observational Studies With Multiple Controls. J Am Stat Assoc 2014. [DOI: 10.1080/01621459.2013.879261] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
|
14
|
Hsu JY, Small DS, Rosenbaum PR. Effect Modification and Design Sensitivity in Observational Studies. J Am Stat Assoc 2013. [DOI: 10.1080/01621459.2012.742018] [Citation(s) in RCA: 24] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/08/2023]
|
15
|
Rosenbaum PR. Impact of multiple matched controls on design sensitivity in observational studies. Biometrics 2013; 69:118-27. [PMID: 23379587 DOI: 10.1111/j.1541-0420.2012.01821.x] [Citation(s) in RCA: 33] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
Abstract
In an observational study, one treated subject may be matched for observed covariates to either one or several untreated controls. The common motivation for using several controls rather than one is to increase the power of a test of no effect under the doubtful assumption that matching for observed covariates suffices to remove bias from nonrandom treatment assignment. Does the choice between one or several matched controls affect the sensitivity of conclusions to violations of this doubtful assumption? With continuous responses, it is known that reducing the heterogeneity of matched pair differences reduces sensitivity to unmeasured biases, but increasing the sample size has a highly circumscribed effect on sensitivity to bias. Is the use of several controls rather than one analogous to a reduction in heterogeneity or to an increase in sample size? The issue is examined for Huber's m-statistics, including the t-test, the examination having three components: an example, asymptotic calculations using design sensitivity, and a simulation. Use of multiple controls with continuous responses yields a nontrivial reduction in sensitivity to unmeasured biases. An example looks at lead and cadmium in the blood of smokers from the 2008 National Health and Nutrition Examination Survey. A by-product of the discussion is a new result giving the design sensitivity for the permutation distribution of m-statistics.
Collapse
Affiliation(s)
- Paul R Rosenbaum
- Department of Statistics, University of Pennsylvania, Philadelphia, Pennsylvania 19104-6340, USA.
| |
Collapse
|
16
|
Zubizarreta JR, Neuman M, Silber JH, Rosenbaum PR. Contrasting Evidence Within and Between Institutions That Provide Treatment in an Observational Study of Alternate Forms of Anesthesia. J Am Stat Assoc 2012; 107:901-915. [PMID: 26664027 PMCID: PMC4673003 DOI: 10.1080/01621459.2012.682533] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
Abstract
In a randomized trial, subjects are assigned to treatment or control by the flip of a fair coin. In many nonrandomized or observational studies, subjects find their way to treatment or control in two steps, either or both of which may lead to biased comparisons. By a vague process perhaps affected by proximity or sociodemographic issues, subjects find their way to institutions that provide treatment. Once at such an institution, a second process, perhaps thoughtful and deliberate, assigns individuals to treatment or control. In the current paper, the institutions are hospitals, and the treatment under study is the use of general anesthesia alone versus some use of regional anesthesia during surgery. For a specific operation, the use of regional anesthesia may be typical in one hospital and atypical in another. A new matched design is proposed for studies of this sort, one that creates two types of nonoverlapping matched pairs. Using a new extension of optimal matching with fine balance, pairs of the first type exactly balance treatment assignment across institutions, so each institution appears in the treated group with the same frequency that it appears in the control group; hence, differences between institutions that affect everyone in the same way cannot bias this comparison. Pairs of the second type compare institutions that assign most subjects to treatment and other institutions that assign most subjects to control, so each institution is represented in the treated group if it typically assigns subjects to treatment or alternatively in the control group if it typically assigns subjects to control, and no institution appears in both groups. By and large, in the second type of matched pair, subjects became treated subjects or controls by choosing an institution, not by a thoughtful and deliberate process of selecting subjects for treatment within institutions. The design provides two evidence factors, that is, two tests of the null hypothesis of no treatment effect that are independent when the null hypothesis is true, where each factor is largely unaffected by certain unmeasured biases that could readily invalidate the other factor. The two factors permit separate and combined sensitivity analyses, where the magnitude of bias affecting the two factors may differ. The case of knee surgery in the study of regional versus general anesthesia is considered in detail.
Collapse
Affiliation(s)
- José R Zubizarreta
- Department of Statistics, The Wharton School, University of Pennsylvania, 473 Jon M. Huntsman Hall, 3730 Walnut Street, Philadelphia, PA 19104-6340 USA
| | - Mark Neuman
- Department of Statistics, The Wharton School, University of Pennsylvania, 473 Jon M. Huntsman Hall, 3730 Walnut Street, Philadelphia, PA 19104-6340 USA
| | - Jeffrey H Silber
- Department of Statistics, The Wharton School, University of Pennsylvania, 473 Jon M. Huntsman Hall, 3730 Walnut Street, Philadelphia, PA 19104-6340 USA
| | - Paul R Rosenbaum
- Department of Statistics, The Wharton School, University of Pennsylvania, 473 Jon M. Huntsman Hall, 3730 Walnut Street, Philadelphia, PA 19104-6340 USA
| |
Collapse
|
17
|
Zhang K, Small DS, Lorch S, Srinivas S, Rosenbaum PR. Using Split Samples and Evidence Factors in an Observational Study of Neonatal Outcomes. J Am Stat Assoc 2011. [DOI: 10.1198/jasa.2011.ap10604] [Citation(s) in RCA: 21] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022]
|