1
|
Hoogland J, Efthimiou O, Nguyen TL, Debray TPA. Evaluating individualized treatment effect predictions: A model-based perspective on discrimination and calibration assessment. Stat Med 2024. [PMID: 39090523 DOI: 10.1002/sim.10186] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2022] [Revised: 06/07/2024] [Accepted: 07/16/2024] [Indexed: 08/04/2024]
Abstract
In recent years, there has been a growing interest in the prediction of individualized treatment effects. While there is a rapidly growing literature on the development of such models, there is little literature on the evaluation of their performance. In this paper, we aim to facilitate the validation of prediction models for individualized treatment effects. The estimands of interest are defined based on the potential outcomes framework, which facilitates a comparison of existing and novel measures. In particular, we examine existing measures of discrimination for benefit (variations of the c-for-benefit), and propose model-based extensions to the treatment effect setting for discrimination and calibration metrics that have a strong basis in outcome risk prediction. The main focus is on randomized trial data with binary endpoints and on models that provide individualized treatment effect predictions and potential outcome predictions. We use simulated data to provide insight into the characteristics of the examined discrimination and calibration statistics under consideration, and further illustrate all methods in a trial of acute ischemic stroke treatment. The results show that the proposed model-based statistics had the best characteristics in terms of bias and accuracy. While resampling methods adjusted for the optimism of performance estimates in the development data, they had a high variance across replications that limited their accuracy. Therefore, individualized treatment effect models are best validated in independent data. To aid implementation, a software implementation of the proposed methods was made available in R.
Collapse
Affiliation(s)
- J Hoogland
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
- Epidemiology and Data Science, Amsterdam University Medical Center, Amsterdam, The Netherlands
| | - O Efthimiou
- Institute of Primary Health Care (BIHAM), University of Bern, Bern, Switzerland
- Institute of Social and Preventive Medicine (ISPM), University of Bern, Bern, Switzerland
| | - T L Nguyen
- Section of Epidemiology, Department of Public Health, University of Copenhagen, Copenhagen, Denmark
| | - T P A Debray
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
- Smart Data Analysis and Statistics B.V., Utrecht, The Netherlands
| |
Collapse
|
2
|
Portuguese AJ, Yeh AC, Banerjee R, Holmberg L, Wuliji N, Green DJ, Mielcarek M, Gopal AK, Gooley T, Stevenson P, Cowan AJ. Optimizing Autologous Stem Cell Transplantation in Multiple Myeloma: The Impact of Intensive Chemomobilization. Transplant Cell Ther 2024; 30:774.e1-774.e12. [PMID: 38768908 PMCID: PMC11296896 DOI: 10.1016/j.jtct.2024.05.016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2024] [Revised: 04/23/2024] [Accepted: 05/15/2024] [Indexed: 05/22/2024]
Abstract
Most transplant-eligible multiple myeloma (MM) patients undergo autologous peripheral blood stem cell collection (PBSC) using G-CSF with on-demand plerixafor (G ± P). Chemomobilization (CM) can be used as a salvage regimen after G ± P failure or for debulking residual tumor burden ahead of autologous peripheral blood stem cell transplantation (ASCT). Prior studies utilizing cyclophosphamide-based CM have not shown long-term benefits. At our center, intensive CM (ICM) using a PACE- or HyperCVAD-based regimen has been used to mitigate "excessive" residual disease based on plasma cell (PC) burden or MM-related biomarkers. Given the lack of efficacy of non-ICM, we sought to determine the impact of ICM on event-free survival (EFS), defined as death, progressive disease, or unplanned treatment escalation. We performed a retrospective study of newly diagnosed MM patients who collected autologous PBSCs with the intent to proceed immediately to ASCT at our center between 7/2020 and 2/2023. Patients were excluded if they underwent a tandem autologous or sequential autologous-allogeneic transplant, had primary PC leukemia, received non-ICM treatment (i.e., cyclophosphamide and/or etoposide), or had previously failed G ± P mobilization. To appropriately evaluate the impact of ICM among those who potentially could have received it, we utilized a propensity score matching (PSM) approach whereby ICM patients were compared to a cohort of non-CM patients matched on pre-ASCT factors most strongly associated with the receipt of ICM. Of 451 patients identified, 61 (13.5%) received ICM (PACE-based, n = 45; hyper-CVAD-based, n = 16). Post-ICM/pre-ASCT, 11 patients (18%) required admission for neutropenic fever and/or infection. Among 51 evaluable patients, the overall response rate was 31%; however, 46 of 55 evaluable patients (84%) saw a reduction in M-spike and/or involved free light chains. Among those evaluated with longitudinal peripheral blood flow cytometry (n = 8), 5 patients (63%) cleared circulating blood PCs post-ICM. Compared to patients mobilized with non-CM, ICM patients collected a slightly greater median number of CD34+ cells (10.8 versus 10.2 × 10⁶/kg, P = .018). The median follow-up was 30.6 months post-ASCT. In a PSM multivariable analysis, ICM was associated with significantly improved EFS (hazard ratio [HR] 0.30, 95% CI 0.14 to 0.67, P = .003), but not improved OS (HR 0.38, 95% CI 0.10 to 1.44, P = .2). ICM was associated with longer post-ASCT inpatient duration (+4.1 days, 95% CI, 2.4 to 5.8, P < .001), more febrile days (+0.96 days, 95% CI 0.50 to 1.4, P < .001), impaired platelet engraftment (HR 0.23, 95% CI 0.06 to 0.87, P = .031), more bacteremia (OR 3.41, 95% CI 1.20 to 9.31, P = .018), and increased antibiotic usage (cefepime: +2.3 doses, 95% CI 0.39 to 4.1, P = .018; vancomycin: +1.0 doses, 95% CI 0.23 to 1.8, P = .012). ICM was independently associated with improved EFS in a matched analysis involving MM patients with excessive disease burden at pre-ASCT workup. This benefit came at the cost of longer inpatient duration, more febrile days, greater incidence of bacteremia, and increased antibiotic usage in the immediate post-ASCT setting. Our findings suggest that ICM could be considered for a subset of MM patients, but its use must be weighed carefully against additional toxicity.
Collapse
Affiliation(s)
- Andrew J Portuguese
- Fred Hutchinson Cancer Center, Seattle, Washington; University of Washington, Seattle, Washington.
| | - Albert C Yeh
- Fred Hutchinson Cancer Center, Seattle, Washington; University of Washington, Seattle, Washington
| | - Rahul Banerjee
- Fred Hutchinson Cancer Center, Seattle, Washington; University of Washington, Seattle, Washington
| | - Leona Holmberg
- Fred Hutchinson Cancer Center, Seattle, Washington; University of Washington, Seattle, Washington
| | - Natalie Wuliji
- Fred Hutchinson Cancer Center, Seattle, Washington; University of Washington, Seattle, Washington
| | - Damian J Green
- Fred Hutchinson Cancer Center, Seattle, Washington; University of Washington, Seattle, Washington
| | - Marco Mielcarek
- Fred Hutchinson Cancer Center, Seattle, Washington; University of Washington, Seattle, Washington
| | - Ajay K Gopal
- Fred Hutchinson Cancer Center, Seattle, Washington; University of Washington, Seattle, Washington
| | - Ted Gooley
- Fred Hutchinson Cancer Center, Seattle, Washington
| | | | - Andrew J Cowan
- Fred Hutchinson Cancer Center, Seattle, Washington; University of Washington, Seattle, Washington
| |
Collapse
|
3
|
Brumberg K, Small DS, Rosenbaum PR. Optimal refinement of strata to balance covariates. Biometrics 2024; 80:ujae061. [PMID: 38994639 DOI: 10.1093/biomtc/ujae061] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2023] [Revised: 05/21/2024] [Accepted: 06/18/2024] [Indexed: 07/13/2024]
Abstract
What is the best way to split one stratum into two to maximally reduce the within-stratum imbalance in many covariates? We formulate this as an integer program and approximate the solution by randomized rounding of a linear program. A linear program may assign a fraction of a person to each refined stratum. Randomized rounding views fractional people as probabilities, assigning intact people to strata using biased coins. Randomized rounding is a well-studied theoretical technique for approximating the optimal solution of certain insoluble integer programs. When the number of people in a stratum is large relative to the number of covariates, we prove the following new results: (i) randomized rounding to split a stratum does very little randomizing, so it closely resembles the linear programming relaxation without splitting intact people; (ii) the linear relaxation and the randomly rounded solution place lower and upper bounds on the unattainable integer programming solution; and because of (i), these bounds are often close, thereby ratifying the usable randomly rounded solution. We illustrate using an observational study that balanced many covariates by forming matched pairs composed of 2016 patients selected from 5735 using a propensity score. Instead, we form 5 propensity score strata and refine them into 10 strata, obtaining excellent covariate balance while retaining all patients. An R package optrefine at CRAN implements the method. Supplementary materials are available online.
Collapse
Affiliation(s)
- Katherine Brumberg
- Department of Statistics, University of Michigan, Ann Arbor, MI 48109, United States
| | - Dylan S Small
- Department of Statistics and Data Science, University of Pennsylvania, Philadelphia, PA 19104, United States
| | - Paul R Rosenbaum
- Department of Statistics and Data Science, University of Pennsylvania, Philadelphia, PA 19104, United States
| |
Collapse
|
4
|
Layrisse-Landaeta V, Dincheva GR, Khedr S, Geng A, Schombs M, Maisha K, Chao SY, Chen CC. Impact of English proficiency on use of Glasgow Coma Scale in geriatric patients with traumatic brain injury. Trauma Surg Acute Care Open 2024; 9:e001439. [PMID: 38957620 PMCID: PMC11217998 DOI: 10.1136/tsaco-2024-001439] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2024] [Accepted: 06/02/2024] [Indexed: 07/04/2024] Open
Abstract
Background The relationship between English proficiency (EP), Glasgow Coma Scale (GCS) and traumatic brain injury (TBI) is not well characterized. We aimed to understand the impact of limited English proficiency (LEP) on the evaluation and outcomes of TBI. Methods Retrospective comparative study in a single institution of patients aged ⪰65 who presented to the emergency department after a fall with head strike between January 2018 and December 2021. TBI was defined as documented loss of consciousness or intracranial hemorrhage (ICH). Relationships between EP, GCS, and TBI were analyzed with multivariable and propensity score-matched models. Results Of the 2905 included, 1233 (42%) had LEP. Most LEP patients were Asian (60%) while the majority of EP patients were non-Hispanic Caucasians (72%). In a univariate analysis, LEP had higher incidence of decreased GCS and was strongly correlated with risk of TBI (OR 1.47, CI 1.26 to 1.71). After adjusting for multiple covariates including race, LEP did not have a significantly increased risk for GCS score <13 (OR 1.66, CI 0.99 to 2.76) or increased risk of TBI. In the matched analysis, LEP had a small but significantly higher risk of GCS score <13 (OR 1.03, CI 1.02 to 1.05) without an increased risk in TBI. Decreased GCS remained strongly correlated with presence of ICH in LEP patients in the adjusted model (OR 1.39, CI 1.30 to 1.50). Conclusions LEP correlated with lower GCS in geriatric patients with TBI. This association weakened after adjusting for factors like race, suggesting racial disparities may have more influence than language differences. Moreover, GCS remained effective for predicting ICH in LEP individuals, highlighting its value with suitable translation resources. Level of evidence This is a Level III evidence restrospective comparative study.
Collapse
Affiliation(s)
| | | | - Shahenda Khedr
- General Surgery, NewYork-Presbyterian Queens, Queens, New York, USA
| | - Andrew Geng
- General Surgery, NewYork-Presbyterian Queens, Queens, New York, USA
| | - Michele Schombs
- General Surgery, NewYork-Presbyterian Queens, Queens, New York, USA
| | - Kazi Maisha
- General Surgery, NewYork-Presbyterian Queens, Queens, New York, USA
| | - Steven Y Chao
- General Surgery, NewYork-Presbyterian Queens, Queens, New York, USA
- Weill Cornell Medicine, New York, New York, USA
| | - Chun-Cheng Chen
- General Surgery, TriStar Skyline Medical Center, Nashville, Tennessee, USA
| |
Collapse
|
5
|
Kiuchi K, Kang X, Nishimura R, Sasayama M, Matsumoto K. Causal Effects of High Stress Assessed Via Interviews on Mental and Physical Health: Toward Computer Agent-Driven Stress Assessment. J Occup Environ Med 2024; 66:e285-e295. [PMID: 38603579 DOI: 10.1097/jom.0000000000003117] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/13/2024]
Abstract
OBJECTIVES This study investigated the causal effect of high stress assessment via an interview on the mental and physical health of workers 1 month later. METHODS Stress assessment interviews and feedback were conducted with 50 Japanese workers. In addition to the interviewer, two occupational health professionals assessed participants' stress based on recordings. The average treatment effect was estimated by propensity score matching. RESULTS High stress, according to the interview-based assessment, had a significant negative causal effect on self-reported well-being 1 month later (95% confidence interval: -3.02, -1.10). In addition, no effect of high stress on stress load, mental and physical symptoms, or burnout was observed. CONCLUSIONS This study provides important insights into the prognosis of individuals who were assessed through interviews to have high stress. The findings are expected to help automate stress assessments using computer agents.
Collapse
Affiliation(s)
- Keita Kiuchi
- From the National Institute of Occupational Safety and Health Japan, Organization of Occupational Health and Safety, Kawasaki, Japan (K.K.); Graduate School of Technology, Industrial and Social Sciences, Tokushima University, Tokushima, Japan (X.K., R.N., K.M.); and National Institute of Technology, Kagawa College, Takamatsu, Japan (M.S.)
| | | | | | | | | |
Collapse
|
6
|
Muhammad T, Srivastava S, Muneera K, Kumar M, Kelekar U. Treatment for Insomnia Symptoms is Associated with Reduced Depression Among Older Adults: A Propensity Score Matching Approach. Clin Gerontol 2024; 47:436-451. [PMID: 37153958 DOI: 10.1080/07317115.2023.2208582] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 05/10/2023]
Abstract
OBJECTIVES The study aimed to investigate the effect of utilization of treatment for insomnia symptoms on the prevalence of major depressive disorder among older adults in India. METHODS We used the data from the Longitudinal Ageing Study in India (LASI), 2017-18. The sample included 10,911 older individuals who reported insomnia symptoms. The propensity score matching (PSM) approach was used to compare the depressive disorder among those who received vs. not received treatment. RESULTS Only 5.7% of older adults reporting insomnia symptoms received treatment. On average, prevalence of depressive disorder among men and women who received treatment for insomnia symptoms was lesser by 0.79 and 0.33 points, respectively, than those who did not receive treatment. In the matched sample, treatment for insomnia symptoms was significantly associated with lesser prevalence of depression for both older men (β= -0.68, p < .001) and older women (β= -0.62, p < .001). CONCLUSIONS The current findings suggest that treatment for insomnia symptoms can reduce the risk of depressive disorder among older adults and the effects are higher among older men than women.
Collapse
Affiliation(s)
- T Muhammad
- Department of Family & Generations, International Institute for Population Sciences, Mumbai, India
| | - Shobhit Srivastava
- Department of Survey Research & Data Analytics, International Institute for Population Sciences, Mumbai, India
| | - K Muneera
- School of Management Studies, National Institute of Technology, Calicut, Kerala, India
| | - Manish Kumar
- Population Research Centre, Dharwad, Karnataka, India
| | - Uma Kelekar
- School of Business, College of Business, Innovation, Leadership and Technology
- Marymount Center for Optimal Aging, Marymount University, Arlington-VA, USA
| |
Collapse
|
7
|
Zhou H, Zhang H, Xiao AY, Yu L, Tang K. Effects of an Animation-Based Comprehensive Sexuality Education Package: A 2-Year Repeated Intervention Study. HEALTH EDUCATION & BEHAVIOR 2024:10901981241236415. [PMID: 38634397 DOI: 10.1177/10901981241236415] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/19/2024]
Abstract
BACKGROUND Preadolescents, who are in a transitional phase of development, may experience higher exposure to heterosexual interactions while facing higher risks regarding misinformation in sexual knowledge and unsafe engagement in sexual activities. There is a deficiency in the availability of qualified educators and age-appropriate teaching materials for sexuality education in China. METHODS We implemented an animation-based comprehensive sexuality education package among preadolescents aged 9 to 12 years from eight schools in Anhui, China. The first round of intervention included 1,835 participants, lasting 2 months from September to November 2020. A total of 374 participants, accounting for 52% of the intervention group, received a second round of intervention in September 2021. Participants completed immediate follow-up assessment and 1-year follow-up assessment to assess changes in their sexual knowledge, attitudes, and other outcomes. Propensity score matching and difference-in-difference analysis were performed to determine the short- and long-term impacts. RESULTS Significant improvements were observed for both sexual knowledge and sexual attitudes in the immediate follow-up. There was no significant effect on pornography-seeking behavior or awareness of experiencing sexual abuse. After 1 year, the effect was sustained for sexual knowledge, but slightly declined for sexual attitudes. The second intervention significantly improved sexual knowledge; however, no significant change in sexual attitudes, pornography-seeking behavior, or awareness of experiencing sexual abuse was observed. CONCLUSIONS Our comprehensive sexuality education package was effective in improving sexual knowledge both immediately and 1 year after the intervention. Repeated intervention can be an effective strategy for promoting preadolescent health development regarding comprehensive sexuality education.
Collapse
Affiliation(s)
- Hui Zhou
- Tsinghua University, Beijing, China
- Harvard University, Boston, MA, USA
| | | | | | - Lei Yu
- Jinan Vocational College of Nursing, Shandong, China
| | - Kun Tang
- Tsinghua University, Beijing, China
| |
Collapse
|
8
|
Nepal M, Ashfaq M, Sharma BR, Shrestha MS, Khadgi VR, Bruno Soares M. Impact of weather and climate advisories on agricultural outcomes in Pakistan. Sci Rep 2024; 14:1036. [PMID: 38200123 PMCID: PMC10781742 DOI: 10.1038/s41598-023-51066-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2023] [Accepted: 12/30/2023] [Indexed: 01/12/2024] Open
Abstract
The earth is warming, and the frequency of extreme weather events have been rapidly growing globally with unprecedented consequences to farming communities. In principle, weather and climate information services (WCIS) can help farmers better manage their activities and farm level outcomes by supporting their decision-making with relevant and usable information to address the potential impacts of expected changing climate conditions. But, in practice, can the availability and use of WCIS help improve agricultural outcomes given the weather and climate related uncertainties? To understand the use and impact of WCIS in the cotton-wheat cropping areas of Pakistan, we conducted a multistage stratified clustered random sample of 612 farm households in the provinces of Punjab and Sindh. Over 55% of the farm households in the sample indicated that they used WCIS provided by the Pakistan Meteorological Department and other sources for making their agricultural decisions. Our analysis, however, suggests that the impact of using WCIS on major farm outcomes (i.e. farm revenue, costs, profits, and input usage) is not statistically significant when compared with those farm households not using WCIS (null result). This result is robust to different estimation techniques (i.e. ordinary least squares, instrumental variable approach, and propensity score matching method). From the focus group discussions, we gathered that farmers perceived WCIS as less reliable, often unclear, and difficult to understand as this information is not translated and transmitted in local languages. Addressing these issues can help reduce the impact of climate and weather variability on farm outcomes in those provinces as well as in Pakistan more generally. Our study suggests that, under uncertainty, emphasis should be on WCIS that farmers can rely on for making farming related decisions.
Collapse
Affiliation(s)
- Mani Nepal
- South Asian Network for Development and Environmental Economics (SANDEE), GPO Box 3226, Kathmandu, Nepal.
- International Center for Integrated Mountain Development (ICIMOD), Lalitpur, Nepal.
| | - Muhammad Ashfaq
- Muhamad Nawaz Shareef University of Agriculture, Multan, Pakistan
| | - Bibek Raj Sharma
- South Asian Network for Development and Environmental Economics (SANDEE), GPO Box 3226, Kathmandu, Nepal
- International Center for Integrated Mountain Development (ICIMOD), Lalitpur, Nepal
| | | | - Vijay Ratan Khadgi
- International Center for Integrated Mountain Development (ICIMOD), Lalitpur, Nepal
| | | |
Collapse
|
9
|
Heinz P, Wendel-Garcia PD, Held U. Impact of the matching algorithm on the treatment effect estimate: A neutral comparison study. Biom J 2024; 66:e2100292. [PMID: 35385172 DOI: 10.1002/bimj.202100292] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2021] [Revised: 02/22/2022] [Accepted: 02/25/2022] [Indexed: 11/12/2022]
Abstract
Propensity score matching is increasingly being used in the medical literature. Choice of matching algorithms, reporting quality, and estimands are oftentimes not discussed. We evaluated the impact of propensity score matching algorithms, based on a recent clinical dataset, with three commonly used outcomes. The resulting estimands for different strengths of treatment effects were compared in a neutral comparison study and based on a thoroughly designed simulation study. Different algorithms yielded different levels of balance after matching. Along with full matching and genetic matching with replacement, good balance was achieved with nearest neighbor matching with caliper but thereby more than one fifth of the treated units were discarded. Average marginal treatment effect estimates were least biased with genetic or nearest neighbor matching, both with replacement and full matching. Double adjustment yielded conditional treatment effects that were closer to the true values, throughout. The choice of the matching algorithm had an impact on covariate balance after matching as well as treatment effect estimates. In comparison, genetic matching with replacement yielded better covariate balance than all other matching algorithms. A literature review in the British Medical Journal including its subjournals revealed frequent use of propensity score matching; however, the use of different matching algorithms before treatment effect estimation was only reported in one out of 21 studies. Propensity score matching is a methodology for causal treatment effect estimation from observational data; however, the methodological difficulties and low reporting quality in applied medical research need to be addressed.
Collapse
Affiliation(s)
- Priska Heinz
- Epidemiology, Biostatistics and Prevention Institute, Department of Biostatistics, University of Zurich, Zurich, Switzerland
| | | | - Ulrike Held
- Epidemiology, Biostatistics and Prevention Institute, Department of Biostatistics, University of Zurich, Zurich, Switzerland
| |
Collapse
|
10
|
Fogarty CB. Testing weak nulls in matched observational studies. Biometrics 2023; 79:2196-2207. [PMID: 35980014 DOI: 10.1111/biom.13741] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2021] [Accepted: 08/01/2022] [Indexed: 11/29/2022]
Abstract
We develop sensitivity analyses for the sample average treatment effect in matched observational studies while allowing unit-level treatment effects to vary. The methods may be applied to studies using any optimal without-replacement matching algorithm. In contrast to randomized experiments and to paired observational studies, we show for general matched designs that over a large class of test statistics, any procedure bounding the worst-case expectation while allowing for arbitrary effect heterogeneity must be unnecessarily conservative if treatment effects are actually constant across individuals. We present a sensitivity analysis which bounds the worst-case expectation while allowing for effect heterogeneity, and illustrate why it is generally conservative if effects are constant. An alternative procedure is presented that is asymptotically sharp if treatment effects are constant, and that is valid for testing the sample average effect under additional restrictions which may be deemed benign by practitioners. Simulations demonstrate that this alternative procedure results in a valid sensitivity analysis for the weak null hypothesis under a host of reasonable data-generating processes. The procedures allow practitioners to assess robustness of estimated sample average treatment effects to hidden bias while allowing for effect heterogeneity in matched observational studies.
Collapse
Affiliation(s)
- Colin B Fogarty
- Operations Research and Statistics Group, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA
| |
Collapse
|
11
|
Zhang B, Mackay EJ, Baiocchi M. Statistical matching and subclassification with a continuous dose: Characterization, algorithm, and application to a health outcomes study. Ann Appl Stat 2023. [DOI: 10.1214/22-aoas1635] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/26/2023]
Affiliation(s)
- Bo Zhang
- Vaccine and Infectious Disease Division, Fred Hutchinson Cancer Center
| | - Emily J. Mackay
- Department of Anesthesiology and Critical Care, Perelman School of Medicine, University of Pennsylvania
| | - Mike Baiocchi
- Department of Epidemiology and Population Health, Stanford University
| |
Collapse
|
12
|
Olmuş H, Beşpınar E, Nazman E. Performance evaluation of some propensity score matching methods by using binary logistic regression model. COMMUN STAT-SIMUL C 2022. [DOI: 10.1080/03610918.2019.1679181] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
Affiliation(s)
- Hülya Olmuş
- Department of Statistics, Faculty of Science, Gazi University, Ankara, Turkey
| | - Esra Beşpınar
- Graduate School of Natural and Applied Sciences, Gazi University, Ankara, Turkey
| | - Ezgi Nazman
- Department of Statistics, Faculty of Science, Gazi University, Ankara, Turkey
| |
Collapse
|
13
|
Bottigliengo D, Baldi I, Lanera C, Lorenzoni G, Bejko J, Bottio T, Tarzia V, Carrozzini M, Gerosa G, Berchialla P, Gregori D. Oversampling and replacement strategies in propensity score matching: a critical review focused on small sample size in clinical settings. BMC Med Res Methodol 2021; 21:256. [PMID: 34809559 PMCID: PMC8609749 DOI: 10.1186/s12874-021-01454-z] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/14/2021] [Accepted: 10/26/2021] [Indexed: 12/03/2022] Open
Abstract
Background Propensity score matching is a statistical method that is often used to make inferences on the treatment effects in observational studies. In recent years, there has been widespread use of the technique in the cardiothoracic surgery literature to evaluate to potential benefits of new surgical therapies or procedures. However, the small sample size and the strong dependence of the treatment assignment on the baseline covariates that often characterize these studies make such an evaluation challenging from a statistical point of view. In such settings, the use of propensity score matching in combination with oversampling and replacement may provide a solution to these issues by increasing the initial sample size of the study and thus improving the statistical power that is needed to detect the effect of interest. In this study, we review the use of propensity score matching in combination with oversampling and replacement in small sample size settings. Methods We performed a series of Monte Carlo simulations to evaluate how the sample size, the proportion of treated, and the assignment mechanism affect the performances of the proposed approaches. We assessed the performances with overall balance, relative bias, root mean squared error and nominal coverage. Moreover, we illustrate the methods using a real case study from the cardiac surgery literature. Results Matching without replacement produced estimates with lower bias and better nominal coverage than matching with replacement when 1:1 matching was considered. In contrast to that, matching with replacement showed better balance, relative bias, and root mean squared error than matching without replacement for increasing levels of oversampling. The best nominal coverage was obtained by using the estimator that accounts for uncertainty in the matching procedure on sets of units obtained after matching with replacement. Conclusions The use of replacement provides the most reliable treatment effect estimates and that no more than 1 or 2 units from the control group should be matched to each treated observation. Moreover, the variance estimator that accounts for the uncertainty in the matching procedure should be used to estimate the treatment effect. Supplementary Information The online version contains supplementary material available at 10.1186/s12874-021-01454-z.
Collapse
Affiliation(s)
- Daniele Bottigliengo
- Unit of Biostatistics, Epidemiology and Public Health, Department of Cardiac, Thoracic, Vascular Sciences and Public Health, University of Padova, Via Loredan 18, 35121, Padova, Italy
| | - Ileana Baldi
- Unit of Biostatistics, Epidemiology and Public Health, Department of Cardiac, Thoracic, Vascular Sciences and Public Health, University of Padova, Via Loredan 18, 35121, Padova, Italy
| | - Corrado Lanera
- Unit of Biostatistics, Epidemiology and Public Health, Department of Cardiac, Thoracic, Vascular Sciences and Public Health, University of Padova, Via Loredan 18, 35121, Padova, Italy
| | - Giulia Lorenzoni
- Unit of Biostatistics, Epidemiology and Public Health, Department of Cardiac, Thoracic, Vascular Sciences and Public Health, University of Padova, Via Loredan 18, 35121, Padova, Italy
| | - Jonida Bejko
- Department of Cardiac, Thoracic,Vascular Sciences and Public Health, University of Padova, Padova, Italy
| | - Tomaso Bottio
- Department of Cardiac, Thoracic,Vascular Sciences and Public Health, University of Padova, Padova, Italy
| | - Vincenzo Tarzia
- Department of Cardiac, Thoracic,Vascular Sciences and Public Health, University of Padova, Padova, Italy
| | - Massimiliano Carrozzini
- Department of Cardiac, Thoracic,Vascular Sciences and Public Health, University of Padova, Padova, Italy
| | - Gino Gerosa
- Department of Cardiac, Thoracic,Vascular Sciences and Public Health, University of Padova, Padova, Italy
| | - Paola Berchialla
- Department of Clinical and Biological Sciences, University of Torino, Torino, Italy
| | - Dario Gregori
- Unit of Biostatistics, Epidemiology and Public Health, Department of Cardiac, Thoracic, Vascular Sciences and Public Health, University of Padova, Via Loredan 18, 35121, Padova, Italy.
| |
Collapse
|
14
|
Jreich R, Sebastien B. Comparison of statistical methodologies used to estimate the treatment effect on time-to-event outcomes in observational studies. J Biopharm Stat 2021; 31:469-489. [PMID: 34403296 DOI: 10.1080/10543406.2021.1918140] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
Abstract
The use of real-world data became more and more popular in the pharmaceutical industry. The impact of real-world evidence is now well emphasized by the regulatory authorities. Indeed, the analysis of this type of data can play a key role for treatment efficacy and safety. The aim of this work is to assess various methods and give guidance on the comparisons of drugs, mostly with respect to time-to-event data, in non-randomized studies with potentially confounding variables. For that purpose, several statistical methodologies are compared based on simulation studies. These methodologies belong to family classes of methods that are widely used for this type of problem: regression, matching, weighting and subclassification methods. The evaluation criteria used to compare methods performances are the relative bias, the mean square error, the coverage probability and the width of the confidence interval. In this paper, we consider different scenarios of dataset features in order to study the effect of the sample size, the number of covariates and the magnitude of the treatment effect on the statistical methodologies performances. These statistical analyses are conducted within a proportional hazard model framework. Furthermore, we highlight the advantage of using techniques to identify relevant covariates for time-to-event outcomes by comparing two variable selection methods under a frequentist and a Bayesian inference. Based on simulation results, recommendations on each of the family of methods are provided to guide decision making.
Collapse
Affiliation(s)
- Rana Jreich
- R&D Data and Data Science, Clinical Modeling & Evidence Integration, Sanofi
| | - Bernard Sebastien
- R&D Data and Data Science, Clinical Modeling & Evidence Integration, Sanofi
| |
Collapse
|
15
|
Chen J, Ho M, Lee K, Song Y, Fang Y, Goldstein BA, He W, Irony T, Jiang Q, van der Laan M, Lee H, Lin X, Meng Z, Mishra-Kalyani P, Rockhold F, Wang H, White R. The Current Landscape in Biostatistics of Real-World Data and Evidence: Clinical Study Design and Analysis. Stat Biopharm Res 2021. [DOI: 10.1080/19466315.2021.1883474] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
Affiliation(s)
- Jie Chen
- Overland Pharmaceuticals, Inc., Dover, DE
| | | | - Kwan Lee
- Janssen Research and Development, Spring House, PA
| | | | - Yixin Fang
- Global Medical Affairs Statistics, Data and Statistical Sciences, AbbVie, North Chicago, IL
| | - Benjamin A Goldstein
- Duke Clinical Research Institute and Duke University Medical Center, Duke University, Durham, NC
| | - Weili He
- Global Medical Affairs Statistics, Data and Statistical Sciences, AbbVie, North Chicago, IL
| | | | | | | | | | - Xiwu Lin
- Janssen Research and Development, Spring House, PA
| | | | | | - Frank Rockhold
- Global Medical Affairs Statistics, Data and Statistical Sciences, AbbVie, North Chicago, IL
| | - Hongwei Wang
- Global Medical Affairs Statistics, Data and Statistical Sciences, AbbVie, North Chicago, IL
| | | |
Collapse
|
16
|
Lee HJ, Wong JB, Jia B, Qi X, DeLong ER. Empirical use of causal inference methods to evaluate survival differences in a real-world registry vs those found in randomized clinical trials. Stat Med 2020; 39:3003-3021. [PMID: 32643219 PMCID: PMC9813951 DOI: 10.1002/sim.8581] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2018] [Revised: 04/27/2020] [Accepted: 04/30/2020] [Indexed: 01/11/2023]
Abstract
With heighted interest in causal inference based on real-world evidence, this empirical study sought to understand differences between the results of observational analyses and long-term randomized clinical trials. We hypothesized that patients deemed "eligible" for clinical trials would follow a different survival trajectory from those deemed "ineligible" and that this factor could partially explain results. In a large observational registry dataset, we estimated separate survival trajectories for hypothetically trial-eligible vs ineligible patients under both coronary artery bypass surgery (CABG) and percutaneous coronary intervention (PCI). We also explored whether results would depend on the causal inference method (inverse probability of treatment weighting vs optimal full propensity matching) or the approach to combine propensity scores from multiple imputations (the "across" vs "within" approaches). We found that, in this registry population of PCI/CABG multivessel patients, 32.5% would have been eligible for contemporaneous RCTs, suggesting that RCTs enroll selected populations. Additionally, we found treatment selection bias with different distributions of propensity scores between PCI and CABG patients. The different methodological approaches did not result in different conclusions. Overall, trial-eligible patients appeared to demonstrate at least marginally better survival than ineligible patients. Treatment comparisons by eligibility depended on disease severity. Among trial-eligible three-vessel diseased and trial-ineligible two-vessel diseased patients, CABG appeared to have at least a slight advantage with no treatment difference otherwise. In conclusion, our analyses suggest that RCTs enroll highly selected populations, and our findings are generally consistent with RCTs but less pronounced than major registry findings.
Collapse
Affiliation(s)
- Hui-Jie Lee
- Duke University School of Medicine, Department of Biostatistics and Bioinformatics
| | - John B Wong
- Tufts Medical Center, Division of Clinical Decision Making, Department of Medicine, Tufts University School of Medicine
| | - Beilin Jia
- Duke University School of Medicine, Department of Biostatistics and Bioinformatics
| | - Xinyue Qi
- Duke University School of Medicine, Department of Biostatistics and Bioinformatics
| | - Elizabeth R. DeLong
- Duke University School of Medicine, Department of Biostatistics and Bioinformatics
| |
Collapse
|
17
|
Fredrickson MM, Errickson J, Hansen BB. Comment: Matching Methods for Observational Studies Derived from Large Administrative Databases. Stat Sci 2020. [DOI: 10.1214/19-sts740] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
|
18
|
Zebrak KA, Campione JR. The Effect of National Family Caregiver Support Program Services on Caregiver Burden. J Appl Gerontol 2020; 40:963-971. [PMID: 31971062 DOI: 10.1177/0733464819901094] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Abstract
Extant evidence on the effectiveness of caregiver programs in alleviating caregiver burden is mixed, underscoring the need for further investigations. This study evaluated the effect of the National Family Caregiver Support Program (NFCSP) educational services and respite care on caregiver burden. We used survey data from caregivers assigned to program (n = 491) or comparison (n = 417) group based on their reported use of NFCSP services. Adjusted difference-in-differences (DiD) analysis found an increase in mean burden scores for both groups from baseline to 6 or 12 months. Among program caregivers receiving ≥4 hr of NFCSP respite care per week (n = 307) and matched comparisons (n = 370), burden scores decreased slightly for program caregivers (-0.095 points), but increased for comparison caregivers (+0.145 points). The DiD (0.239 points) was not statistically significant. More research is needed to determine the minimum amount of respite care needed to positively impact caregiver burden.
Collapse
|
19
|
Zhang Y, Lu B. Accounting for matching structure in post-matching analysis of observational studies. COMMUN STAT-SIMUL C 2020. [DOI: 10.1080/03610918.2019.1708928] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
Affiliation(s)
- Yuyang Zhang
- Division of Biostatistics, College of Public Health, The Ohio State University, Columbus, OH, USA
| | - Bo Lu
- Division of Biostatistics, College of Public Health, The Ohio State University, Columbus, OH, USA
| |
Collapse
|
20
|
Gutman R, Intrator O, Lancaster T. A Bayesian procedure for estimating the causal effects of nursing home bed-hold policy. Biostatistics 2019; 19:444-460. [PMID: 29028991 DOI: 10.1093/biostatistics/kxx049] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2017] [Accepted: 08/22/2017] [Indexed: 11/13/2022] Open
Abstract
Nursing home bed-hold policies provide continuity of care for Medicaid beneficiaries by paying nursing homes to reserve beds so residents can return to their facility of occupancy following an acute hospitalization. In 2001, Michigan implemented bed-hold policies in nursing homes. We investigated the impact of these policies on mortality and hospitalizations using 1999-2004 quarterly data from nursing homes in Michigan and nursing homes in 11 states that did not implement such policies. Synthetic Control has been used to estimate the effects of policies by accounting for changes over time unrelated to the intervention. Synthetic Control is intended for scalar continuous outcome at each period, and assumes a single treated unit and multiple control units. We propose a Bayesian procedure to overcome these limitations. It imputes the outcomes of nursing homes in Michigan if they were not exposed to the policy by matching to non-exposed nursing homes that are associated with the exposed ones in the pre-policy period. Because sampling from a Bayesian model is computationally challenging, we describe an approximation procedure that can be implemented using existing software. Our approach can be applied to other studies that examine the impact of policies.
Collapse
Affiliation(s)
- Roee Gutman
- Department of Biostatistics, Brown University, 121 South Main Street, Providence, RI, USA
| | - Orna Intrator
- University of Rochester Medical Center, Public Health Sciences, 265 Crittenden Blvd., Rochester, NY, USA
| | - Tony Lancaster
- Department of Economics, Box B, Brown University, Providence, RI, USA
| |
Collapse
|
21
|
Small DS, Sorenson SB, Berk RA. After the gun: examining police visits and intimate partner violence following incidents involving a firearm. J Behav Med 2019; 42:591-602. [PMID: 31367925 DOI: 10.1007/s10865-019-00013-8] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2018] [Accepted: 01/16/2019] [Indexed: 12/11/2022]
Abstract
Laws have been enacted to keep firearms out of the hands of abusers. In this study, we examined one such effort-removal of a firearm at the scene of intimate partner violence (IPV)-to assess the subsequent occurrence and number of IPV incidents responded to by police and subsequent risk of injury to the victim. Using the 28,977 IPV calls in one large U.S. city to which officers responded during the 2013 calendar year, we identified 220 first-time incidents in which offenders used (i.e., brandished, pistol whipped, shot) a pistol, revolver, rifle, or shotgun. Officers reported removing a firearm from 52 (24%) of the offenders. After using full propensity score matching to control for potential confounders, logistic and Poisson regressions were used to assess differences between those from whom a firearm was removed and those whose firearm was not removed. Firearm removal at the scene of an IPV incident appears to increase the likelihood of subsequent IPV reports to police and suggestive evidence that subsequent injury to the victim might increase as well. The offender shifting from threats with a firearm to physical violence and a change (an increase as well as a decrease) in victim willingness to summon police may account for the findings.
Collapse
Affiliation(s)
- Dylan S Small
- Department of Statistics, Wharton School, University of Pennsylvania, Philadelphia, PA, USA
| | - Susan B Sorenson
- School of Social Policy and Practice, University of Pennsylvania, 3815 Walnut Street, Philadelphia, PA, 19104, USA. .,Ortner Center on Violence and Abuse in Relationships, University of Pennsylvania, 3815 Walnut Street, Philadelphia, PA, 19104, USA.
| | - Richard A Berk
- Department of Statistics, Wharton School, University of Pennsylvania, Philadelphia, PA, USA.,Ortner Center on Violence and Abuse in Relationships, University of Pennsylvania, 3815 Walnut Street, Philadelphia, PA, 19104, USA.,Department of Criminology, University of Pennsylvania, Philadelphia, PA, USA
| |
Collapse
|
22
|
Rosenbaum PR. Sensitivity analysis for stratified comparisons in an observational study of the effect of smoking on homocysteine levels. Ann Appl Stat 2018. [DOI: 10.1214/18-aoas1153] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
|
23
|
|
24
|
Lopez MJ, Gutman R. Estimation of Causal Effects with Multiple Treatments: A Review and New Ideas. Stat Sci 2017. [DOI: 10.1214/17-sts612] [Citation(s) in RCA: 63] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
|
25
|
Bu Y, Zhou XH. Statistical evaluation of drug efficacy for bridging study in companion diagnostic test trials. J Biopharm Stat 2016; 26:1118-1124. [PMID: 27649095 DOI: 10.1080/10543406.2016.1226328] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
Abstract
Personalized medicine is an area of growing attention in medical research and practice. A market-ready companion diagnostic test (CDx) is used in personalized medicine for identifying the best treatment for an individual patient. Unfortunately, development of CDx may lag behind the development of the drug, and consequently we use a clinical trial assay (CTA) to enroll patients into the drug pivotal clinical trial instead. Thus, when CDx becomes available, a bridging study will be required to assess the drug efficacy in the CDx intended use (CDx IU) population. Due to missingness of the CDx results that could be associated with randomization, one challenge we face in a bridging study is covariate imbalance between treatment arms for the subpopulation with both positive CDx and CTA. In this paper, we evaluate the performance of two methods in bridging studies under a causal inference framework. Particularly, we aim to use the propensity score method with doubly robust estimation and optimal matching to address the challenge. We extend under a current framework on drug efficacy estimation in the CDx IU population, using data from both the bridging study and the CTA drug pivotal clinical trial. Both approaches are discussed in the context of a randomized bridging study, and a targeted design clinical trial with simulations, followed by analyzing simulated data that mimic a real ongoing clinic trial.
Collapse
Affiliation(s)
- Yunqi Bu
- a Department of Biostatistics , University of Washington , Seattle , Washington , USA
| | - Xiao-Hua Zhou
- a Department of Biostatistics , University of Washington , Seattle , Washington , USA.,b Biostatistics Unit, Seattle-Denver HSR&D Center of Innovation , VA Puget Sound Health Care System , Seattle , Washington , USA
| |
Collapse
|
26
|
Kang H, Kreuels B, May J, Small DS. Full matching approach to instrumental variables estimation with application to the effect of malaria on stunting. Ann Appl Stat 2016. [DOI: 10.1214/15-aoas894] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
|
27
|
Porcher R, Leyrat C, Baron G, Giraudeau B, Boutron I. Performance of principal scores to estimate the marginal compliers causal effect of an intervention. Stat Med 2015; 35:752-67. [DOI: 10.1002/sim.6735] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2015] [Revised: 07/03/2015] [Accepted: 08/27/2015] [Indexed: 11/11/2022]
Affiliation(s)
- Raphaël Porcher
- Université Paris Decartes; Sorbonne Paris Cité Paris UMR-S 1153 France
- Inserm U1153; Paris France
- Assistance Publique-Hôpitaux de Paris; Hôtel-Dieu, Centre d' Épidémiologie Clinique; Paris France
| | - Clémence Leyrat
- Inserm U1153; Paris France
- INSERM CIC 1415; Tours France
- CHRU de Tours; Tours France
| | - Gabriel Baron
- Inserm U1153; Paris France
- Assistance Publique-Hôpitaux de Paris; Hôtel-Dieu, Centre d' Épidémiologie Clinique; Paris France
| | - Bruno Giraudeau
- Inserm U1153; Paris France
- INSERM CIC 1415; Tours France
- CHRU de Tours; Tours France
- Université François-Rabelais, PRES Centre-Val de Loire Université; Tours France
| | - Isabelle Boutron
- Université Paris Decartes; Sorbonne Paris Cité Paris UMR-S 1153 France
- Inserm U1153; Paris France
- Assistance Publique-Hôpitaux de Paris; Hôtel-Dieu, Centre d' Épidémiologie Clinique; Paris France
| |
Collapse
|
28
|
Pimentel SD, Yoon F, Keele L. Variable-ratio matching with fine balance in a study of the Peer Health Exchange. Stat Med 2015; 34:4070-82. [DOI: 10.1002/sim.6593] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/19/2015] [Revised: 06/25/2015] [Accepted: 06/25/2015] [Indexed: 11/08/2022]
Affiliation(s)
| | - Frank Yoon
- Mathematica Policy Research; Princeton NJ U.S.A
| | - Luke Keele
- Penn State University; University Park; PA U.S.A
- American Institutes for Research; Washington D.C. U.S.A
| |
Collapse
|
29
|
Doherty EE, Cwick JM, Green KM, Ensminger ME. Examining the Consequences of the "Prevalent Life Events" of Arrest and Incarceration among an Urban African-American Cohort. JUSTICE QUARTERLY : JQ 2015; 33:970-999. [PMID: 27616814 PMCID: PMC5015121 DOI: 10.1080/07418825.2015.1016089] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/06/2023]
Abstract
The life course perspective has traditionally examined prevalent adult life events, such as marriage and employment, and their potential to redirect offending trajectories. However, for African Americans, the life events of arrest and incarceration are becoming equally prevalent in young adulthood. Therefore, it is critical to understand how these "standard" criminal justice practices, which are designed to deter as well as punish, affect deviance among this population. This study evaluates the long-term consequences of criminal justice intervention on substance use and offending into midlife among an African American community cohort using propensity score matching and multivariate regression analyses. The results largely point to a criminogenic effect of criminal justice intervention on midlife deviance with a particularly strong effect of young adult arrest on rates of violent and property arrest counts into midlife. The theoretical and policy implications of the findings are discussed.
Collapse
Affiliation(s)
- Elaine Eggleston Doherty
- Department of Criminology and Criminal Justice, University of
Missouri – St. Louis, 324 Lucas Hall, One University Boulevard, St.
Louis, MO 63121, 314-516-5033
| | - Jaclyn M. Cwick
- Department of Criminology and Criminal Justice, University of
Missouri – St. Louis, 324 Lucas Hall, One University Boulevard, St.
Louis, MO 63121
| | - Kerry M. Green
- Department of Behavioral and Community Health, University of
Maryland School of Public Health, SPH Building, Room 2375, College Park, MD
20742, 301-405-2524
| | - Margaret E. Ensminger
- Department of Health, Behavior and Society, Johns Hopkins
University School of Public Health, 624 N. Broadway, 7 Floor,
Baltimore, MD 21205, 410-955-2308
| |
Collapse
|
30
|
Thoemmes FJ, Kim ES. A Systematic Review of Propensity Score Methods in the Social Sciences. MULTIVARIATE BEHAVIORAL RESEARCH 2011; 46:90-118. [PMID: 26771582 DOI: 10.1080/00273171.2011.540475] [Citation(s) in RCA: 200] [Impact Index Per Article: 15.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/05/2023]
Abstract
The use of propensity scores in psychological and educational research has been steadily increasing in the last 2 to 3 years. However, there are some common misconceptions about the use of different estimation techniques and conditioning choices in the context of propensity score analysis. In addition, reporting practices for propensity score analyses often lack important details that allow other researchers to confidently judge the appropriateness of reported analyses and potentially to replicate published findings. In this article we conduct a systematic literature review of a large number of published articles in major areas of social science that used propensity scores up until the fall of 2009. We identify common errors in estimation, conditioning, and reporting of propensity score analyses and suggest possible solutions.
Collapse
|
31
|
Stuart EA. Matching methods for causal inference: A review and a look forward. Stat Sci 2010; 25:1-21. [PMID: 20871802 DOI: 10.1214/09-sts313] [Citation(s) in RCA: 2083] [Impact Index Per Article: 148.8] [Reference Citation Analysis] [Abstract] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
Abstract
When estimating causal effects using observational data, it is desirable to replicate a randomized experiment as closely as possible by obtaining treated and control groups with similar covariate distributions. This goal can often be achieved by choosing well-matched samples of the original treated and control groups, thereby reducing bias due to the covariates. Since the 1970's, work on matching methods has examined how to best choose treated and control subjects for comparison. Matching methods are gaining popularity in fields such as economics, epidemiology, medicine, and political science. However, until now the literature and related advice has been scattered across disciplines. Researchers who are interested in using matching methods-or developing methods related to matching-do not have a single place to turn to learn about past and current research. This paper provides a structure for thinking about matching methods and guidance on their use, coalescing the existing research (both old and new) and providing a summary of where the literature on matching methods is now and where it should be headed.
Collapse
Affiliation(s)
- Elizabeth A Stuart
- Johns Hopkins Bloomberg School of Public Health, Department of Mental Health, Department of Biostatistics, 624 N Broadway, 8th Floor, Baltimore, MD 21205
| |
Collapse
|
32
|
Abstract
A predictor variable or dose that is measured with substantial error may possess an error-free milestone, such that it is known with negligible error whether the value of the variable is to the left or right of the milestone. Such a milestone provides a basis for estimating a linear relationship between the true but unknown value of the error-free predictor and an outcome, because the milestone creates a strong and valid instrumental variable. The inferences are nonparametric and robust, and in the simplest cases, they are exact and distribution free. We also consider multiple milestones for a single predictor and milestones for several predictors whose partial slopes are estimated simultaneously. Examples are drawn from the Wisconsin Longitudinal Study, in which a BA degree acts as a milestone for sixteen years of education, and the binary indicator of military service acts as a milestone for years of service.
Collapse
Affiliation(s)
- Dylan S Small
- Department of Statistics, The Wharton School, University of Pennsylvania, 473 Jon M. Huntsman Hall, 3730 Walnut Street, Philadelphia, Pennsylvania 19104-6340, USA
| | | |
Collapse
|
33
|
Hill J, Reiter JP. Interval estimation for treatment effects using propensity score matching. Stat Med 2006; 25:2230-56. [PMID: 16220488 DOI: 10.1002/sim.2277] [Citation(s) in RCA: 81] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
In causal studies without random assignment of treatment, causal effects can be estimated using matched treated and control samples, where matches are obtained using estimated propensity scores. Propensity score matching can reduce bias in treatment effect estimators in cases where the matched samples have overlapping covariate distributions. Despite its application in many applied problems, there is no universally employed approach to interval estimation when using propensity score matching. In this article, we present and evaluate approaches to interval estimation when using propensity score matching.
Collapse
Affiliation(s)
- Jennifer Hill
- School of International and Public Affairs, Columbia University, 420 West 118th St., 740 IAB, New York, NY 10027, USA.
| | | |
Collapse
|
34
|
Hill JL, Waldfogel J, Brooks-Gunn J, Han WJ. Maternal employment and child development: a fresh look using newer methods. Dev Psychol 2006; 41:833-50. [PMID: 16351331 DOI: 10.1037/0012-1649.41.6.833] [Citation(s) in RCA: 63] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
The employment rate for mothers with young children has increased dramatically over the past 25 years. Estimating the effects of maternal employment on children's development is challenged by selection bias and the missing data endemic to most policy research. To address these issues, this study uses propensity score matching and multiple imputation. The authors compare outcomes across 4 maternal employment patterns: no work in first 3 years postbirth, work only after 1st year, part-time work in 1st year, and full-time work in 1st year. Our results demonstrate small but significant negative effects of maternal employment on children's cognitive outcomes for full-time employment in the 1st year postbirth as compared with employment postponed until after the 1st year. Multiple imputation yields noticeably different estimates as compared with a complete case approach for many measures. Differences between results from propensity score approaches and regression modeling are often minimal.
Collapse
Affiliation(s)
- Jennifer L Hill
- School of International and Public Affairs, Columbia University, NY 10027, USA.
| | | | | | | |
Collapse
|